How are you currently utilizing Genie in batch mode for processing user queries, and what benefits have you observed from this approach?
Are there specific strategies or best practices that you have found particularly effective in optimizing the efficiency and accuracy of batch processing with Genie?
Share your experiences and insights on the challenges and successes you've encountered while implementing Genie in batch mode, and let's discuss ways to enhance its impact in our organizations.
Treating the Genie Agent using the asynchronous conversation API workflow will be a good choice here.
If the Genie Agent is incomplete or untested, even the highest-quality data will fail a Genie Agent. For API-driven batching, the most effective operational patterns that I can suggest include the following:
Queuing requests instead of firing them all at once.
Using exponential backoff for retries and polling.
Polling every 1-5 seconds and stopping after a reasonable timeout (e.g., 10 minutes) is also considered best practice.
Thanks Lingeshwaran. Do you have any recommended documentation to go through?
The official conversational API docs : https://docs.databricks.com/aws/en/genie/conversation-api is the starting point for your design.
But good references around MLflow evaluation can be found here: https://mlflow.org/cookbook/genie-tracing-pipeline/ and the next blog in that learning process: https://mlflow.org/cookbook/genie-evaluation-judges/
One of the best practice for accuracy is to start small with the amount of tables you have in your Genie space. There is a limit of 30 tables per Genie space but you may want to start with 5-7 tables so that you can incrementally teach Genie about your data.
