Role Overview – Huemot Technology Pvt. Ltd is hiring a Data Scientist specializing in AI/LLM Evaluation for a full-time, remote position serving clients across the United States. This position is ideal for accomplished data science professionals with strong AI and machine learning expertise, large language model knowledge, and the ability to evaluate AI systems effectively, assess model performance, develop evaluation frameworks, conduct rigorous testing, provide insights, and contribute to organizational success within a dynamic AI technology organization.

LLM Evaluation and Assessment – You will evaluate large language models by assessing performance, testing capabilities, measuring accuracy, and providing comprehensive analysis. You will develop evaluation methodologies and support model improvement. You should conduct thorough LLM evaluation.

Model Performance Measurement – You will measure model performance by tracking metrics, analyzing results, calculating accuracy, and evaluating effectiveness. You will assess model quality and capabilities. You should measure performance comprehensively.

Prompt Engineering and Testing – You will test prompts by engineering effective prompts, evaluating responses, measuring consistency, and identifying optimal approaches. You will optimize prompt performance and effectiveness. You should develop strong prompt engineering skills.

Prompt Optimization – You will optimize prompts by refining wording, testing variations, measuring impact, and improving quality. You will enhance prompt effectiveness and results. You should optimize prompts systematically.

Response Quality Assessment – You will assess response quality by evaluating outputs, measuring coherence, checking accuracy, and providing detailed feedback. You will ensure high-quality LLM responses. You should evaluate response quality thoroughly.

Accuracy and Correctness Evaluation – You will evaluate accuracy by verifying information, checking correctness, measuring precision, and identifying errors. You will ensure model accuracy. You should assess accuracy rigorously.

Consistency Testing – You will test consistency by running repeated evaluations, measuring variance, identifying inconsistencies, and reporting findings. You will assess model consistency and stability. You should evaluate consistency.

How to Apply:

Send your updated CV to sanjana.patel@huemot.com. Please highlight your data science experience and professional achievements, your expertise in AI and machine learning, your specific experience with large language model evaluation, your prompt engineering and optimization skills, your statistical analysis and data analysis capabilities, your Python programming and ML framework proficiency, your experience developing evaluation frameworks and assessment methodologies, your quality assurance and testing expertise, your ability to identify bias, fairness, and safety issues in AI systems, your research and investigation skills, your communication and documentation abilities, your educational background and relevant certifications, your remote work effectiveness and self-direction, your problem-solving approach to technical challenges, and your commitment to staying current with AI advancements.

About the Company:

Huemot Technology Pvt. Ltd is a professional AI and technology organization serving clients across the United States with a strong commitment to AI excellence, rigorous evaluation methodology, and organizational success through effective AI system assessment and optimization. The company is seeking experienced data scientists who bring proven data science expertise and AI/ML mastery, expert large language model knowledge and evaluation capability, strong statistical analysis and data analysis skills, demonstrated ability to develop comprehensive evaluation frameworks, expertise in assessing AI system performance.