Have something to say?
Add Streaming Response for Jupyter/REPL
In Jupyter, showing partial responses is very UX-friendly. Use LangChain's streaming interface to update a cell incrementally.
Add Memory / Conversation History
Right now each question is stateless. Add an optional conversation memory (e.g., past 3 Q&A pairs) in AgentState["messages"], and pass them to the LLM for better coherence.
Better prompts
Currently the prompts are quite vanilla. There is significant room for improvement for better prompt templates.
Async & Parallel Context Gathering
Currently everything is sequential. Instead: Gather traceback, module info, and context summary concurrently. Use async def and asyncio.gather() for performance (especially in Jupyter).
Improved context for large modules
Currently, we add docs for all submodules as context. This can quickly get expensive or even run out of context window. We need to do smart context engineering by using RAG/Summarization/Selective filtering or a combination of these.