SOFTWARE TEST ENGINEER @ IDEAS TO IMPACTS DIGITAL
Ideas to Impacts Digital is seeking a Software Test Engineer with specialized exposure to AI systems to contribute to the validation of next-generation AI applications. This role is central to evaluating AI-powered systems developed using Large Language Models, Retrieval-Augmented Generation, and Agentic AI workflows. The position is based in Pune (Baner) and follows a hybrid work model requiring three days of onsite presence per week. Candidates will join a team dedicated to ensuring the reliability and accuracy of advanced artificial intelligence implementations. Responsibilities include evaluating AI generated responses for accuracy, relevance, and consistency while checking for potential hallucinations. The successful candidate will validate RAG pipelines and document retrieval grounding, alongside testing multi-step reasoning within agent-based AI workflows. The role involves designing structured AI test scenarios, writing Python scripts for system evaluation, and working closely with AI engineers to debug and optimize system behavior. Strong analytical skills and a background in Python and API testing are essential for this technical position.
Key Requirements
Strong proficiency in Python programming for script development.
Direct exposure to LLM-based applications or AI chatbot systems.
Deep understanding of RAG (Retrieval-Augmented Generation) pipelines.
Experience in test case design specifically for AI and API testing.
Ability to evaluate AI responses for accuracy, relevance, and hallucinations.
Skill in validating document retrieval grounding within AI systems.
Experience testing agent-based AI workflows and multi-step reasoning.
Capability to design structured AI test scenarios and complex queries.
Proven experience writing Python scripts for AI evaluation and testing.
Ability to work closely with AI engineers to identify and debug system behavior.
Understanding of hybrid software development and testing cycles.
Knowledge of software quality assurance best practices and methodologies.