Regarding the AI design pattern, we recently proposed Leeroo Orchestrator, which is lightweight LLM that knows when and how to leverage underlying LLM experts to perform a task with the best outcome. Here are some of the outcomes:
State-of-the-art open-source: When leveraging open-source models, the Leeroo Orchestrator establishes itself as the top-performing open-source model on the MMLU benchmark, attaining 76% accuracy — with the same budget as Mixtral (70.6%).
Leeroo open-source vs. GPT3.5: Leeroo Orchestrator open-source achieves GPT3.5 accuracy on the MMLU at almost a fourth of the cost.
Achieving and beyond GPT4 with a fraction of cost: Combining open-source models with GPT4, the Orchestrator nearly matches GPT4’s accuracy at half the cost. Moreover, it even surpasses GPT4’s accuracy with 25% less cost.
Accessible: Leeroo can be served on accessible consumer hardware since it leverages underlying models of small sizes, 7b-34b (e.g., A10 and A100), and can be deployed on any cloud provider or on-prem.
“imagine this idealized metadata platform. It knows everything there is to know about how data flows through your organization and what it means.
Now pair that with the advances in AI: higher-level conceptual reasoning, performance, and system design.”
I wonder how many organizations will be able to make snowflake (or similar) their data store for everything. OR will AI enable a more data lakehouse approach where the AI tools connect to data where it already lives?
Regarding the AI design pattern, we recently proposed Leeroo Orchestrator, which is lightweight LLM that knows when and how to leverage underlying LLM experts to perform a task with the best outcome. Here are some of the outcomes:
State-of-the-art open-source: When leveraging open-source models, the Leeroo Orchestrator establishes itself as the top-performing open-source model on the MMLU benchmark, attaining 76% accuracy — with the same budget as Mixtral (70.6%).
Leeroo open-source vs. GPT3.5: Leeroo Orchestrator open-source achieves GPT3.5 accuracy on the MMLU at almost a fourth of the cost.
Achieving and beyond GPT4 with a fraction of cost: Combining open-source models with GPT4, the Orchestrator nearly matches GPT4’s accuracy at half the cost. Moreover, it even surpasses GPT4’s accuracy with 25% less cost.
Accessible: Leeroo can be served on accessible consumer hardware since it leverages underlying models of small sizes, 7b-34b (e.g., A10 and A100), and can be deployed on any cloud provider or on-prem.
More info in: https://www.leeroo.com/
“imagine this idealized metadata platform. It knows everything there is to know about how data flows through your organization and what it means.
Now pair that with the advances in AI: higher-level conceptual reasoning, performance, and system design.”
I wonder how many organizations will be able to make snowflake (or similar) their data store for everything. OR will AI enable a more data lakehouse approach where the AI tools connect to data where it already lives?
Great! Thanks you for reasoning so clearly. Another future big step would be AI inferencing the metadata, observing...