watsonx Test Forge
Hackathon: auto-generate Journey Success test cases for IBM watsonx agents
Built by Devanshu Chicholikar, software engineer and AI engineer in Boston.
IBM watsonx Orchestrate hackathon. The problem: manually authoring Journey Success test cases for watsonx agents is tedious and error-prone. Test Forge is a multi-tool agent that reads deployed agent specs and generates validated test cases automatically.
Tool lifecycle: list_deployed_agents, get_agent_spec, generate_test_case, upload_test_case. Test cases cover happy path, edge cases, and failure scenarios with strict/fuzzy/optional argument matching plus response text keywords. All goals must pass for a test to succeed.
Built on IBM's ADK + watsonx Orchestrate MCP server, integrated with Bob IDE. Demonstrates manager/collaborator agent composition: manager agents coordinate via named collaborators; collaborator agents own the tools.
- 4
- Tools implemented
- 13
- Unit tests passing
- Hackathon
- IBM ADK
- Journey Success
- Evaluation metric
list, get_spec, generate, upload with ToolResponse wrapping
Schema validation, error handling, tool contracts
Built on IBM ADK and watsonx Orchestrate
Tool-call + text-keyword matching