Bruno.

Bruno.

@bdagnino.com

Building Limai: automated data extraction from unstructured sources | Climate Tech | Ex @PachamaInc | Co-founder @MetricaSports | 🇦🇷🇪🇸

29 followers20 following48 posts

What's the best way to monitor LLMs that use the Gemini API? I used to use langfuse but it doesn't seem work as nicely as it does with openai.

Using Few Shot examples to boost LLM data extraction by over 50%? If you spent countless hours fine-tuning prompts, testing different parsing libraries, and trying to craft perfect solutions only to get mediocre results, this is for you.

In this post you'll learn how: 1. Build a simple benchmark to evaluate the performance of your models 2. How a single in-context examples allowed 4o-mini to out perform 4o 3. How to simple improve model quality, and latency at the same time. Check it out! www.limai.io/blog/example

My 3 mantras to stay sane as an entrepreneur. Always visible in my desk. I should probably have a nicer version framed or something, but hey, who has time for that? 😂

🚀 Preventing Hallucinations in Vision-Based Data Extraction Vision models are coming up as the best way to deal with documents with complex layouts. On the flip side, they are more likely to hallucinate results. How can we address that? With OCR based data validations. 👇

View full thread

The Takeaway Vision-based models are powerful, but validation frameworks are critical for reliable results. 💡 If you’re building data pipelines, combine extraction with validation to ensure accuracy and trust.

🚀 Preventing Hallucinations in Vision-Based Data Extraction Vision models are coming up as the best way to deal with documents with complex layouts. On the flip side, they are more likely to hallucinate results. How can we address that? With OCR based data validations. 👇

View full thread

How It Works • Extract raw text using a PDF reader. • Validate each extracted value (e.g., “160.69 €”) by searching for it in the raw text. • Flag values that don’t match as potential hallucinations.

Key Results ✅ Vision models like Gemini handled layouts flexibly. ✅ Validation caught hallucinations and ensured data accuracy. ✅ Trustworthiness increased for complex documents like utility bills.

🚀 Preventing Hallucinations in Vision-Based Data Extraction Vision models are coming up as the best way to deal with documents with complex layouts. On the flip side, they are more likely to hallucinate results. How can we address that? With OCR based data validations. 👇

View full thread

While vision models excel at "reading" layouts, they sometimes invent data. E.g., instead of extracting "2.983 kW" for contracted power, the model returned "2.0 kW"—a made-up value. 😬 How do we prevent this?

We combined: 1️⃣ Vision-based extraction to handle complex layouts. 2️⃣ Instructor-powered validation to cross-check extracted values against raw text from PDFs. This ensured data was grounded in reality, not hallucinated.

🚀 Preventing Hallucinations in Vision-Based Data Extraction Vision models are coming up as the best way to deal with documents with complex layouts. On the flip side, they are more likely to hallucinate results. How can we address that? With OCR based data validations. 👇

Vision-based extraction is becoming the most promising path forward for Document AI. These models handle complex layouts, tables, and multimodal inputs natively—far beyond what rule-based parsing can achieve. But they also have challenges.

It feels like chess engines are so powerful now that they become a bit useless in chess commentary. Even GMs can't make sense of the eval bar sometimes. It would be better maybe to have a more "human" eval bar that actually helps the audience and commentators.

I love how chess player assign so much meaning, personality, and purpose to chess pieces throughout games. So much passion and emotions on a board game.

🚀 [NEW POST] Show, Don’t Tell: How Dynamic Examples Boosted Accuracy from 61% to 97% Ever spent hours fine-tuning prompts or testing document parsing libraries, only to end up with meh results? What if I told you that one simple change could drastically improve your results?

View full thread

If you’re curious about how this approach can work for you, let’s chat! We’re offering free consulting calls this month to help businesses optimize their AI strategies. 📩 bruno@limai.io or DM me!

🚀 [NEW POST] Show, Don’t Tell: How Dynamic Examples Boosted Accuracy from 61% to 97% Ever spent hours fine-tuning prompts or testing document parsing libraries, only to end up with meh results? What if I told you that one simple change could drastically improve your results?

View full thread

That’s when we tried something so simple it felt obvious in hindsight: we added an example. The results were staggering: • With a small model plus the example, accuracy leaped from 61% to 97%. • We achieved this without fine-tuning or complex parsing techniques.

In our latests post we break down: ✅ How we built a simple test dataset to evaluate accuracy. ✅ Why adding examples worked so well (and why you should try it). ✅ How this influenced our product's UX/UI strategy.

🚀 [NEW POST] Show, Don’t Tell: How Dynamic Examples Boosted Accuracy from 61% to 97% Ever spent hours fine-tuning prompts or testing document parsing libraries, only to end up with meh results? What if I told you that one simple change could drastically improve your results?

View full thread

Czech utility bills. These documents had: 🌎 Non-English text (a hurdle for many LLMs) 🧮 Values that needed to be calculated (e.g., summing multiple rows for Heating or Cooling) 🎲 A mix of other fields like dates, addresses, and contracts details

Even after a lot of work on prompt engineering and trying out parsing libraries our results were stuck at 61%-80% accuracy—not enough for reliable use.

🚀 [NEW POST] Show, Don’t Tell: How Dynamic Examples Boosted Accuracy from 61% to 97% Ever spent hours fine-tuning prompts or testing document parsing libraries, only to end up with meh results? What if I told you that one simple change could drastically improve your results?

While building Limai 's data extraction product, we faced a tough challenge for a proof concept with a potential client: extracting complex data from

Adding and example increased our LLM data extraction accuracy from 61% to 97%. No complex prompt engineering. No fine-tuning. Just showing the model what good output looks like made all the difference. Sometimes the simplest solutions are the most powerful.

It's a funny that LLMs never suggest LLMs as a solution to a problem. Like they are not aware of their own capabilities. I guess it's lack of representation of those solutions in the training data, but still, clear sign that we are not at AGI levels yet 🤣.

🚀 [NEW POST] The Minimum Viable Model (MVM) Framework The most common mistake I saw in more than 10 years of building AI products? overcomplicating things. You can avoid that with these 5 steps and build AI products that actually work. Here's how 🧵👇

View full thread

8/ Free AI consulting calls! At @LimaiConsulting, we’re offering free consulting this month. Want tailored advice on your AI product strategy or how to apply MVM? Let’s chat. 📩 bruno@limai.io or DM me here.

🚀 [NEW POST] The Minimum Viable Model (MVM) Framework The most common mistake I saw in more than 10 years of building AI products? overcomplicating things. You can avoid that with these 5 steps and build AI products that actually work. Here's how 🧵👇

View full thread

5/ Step 4: Use real-world data Early deployment gives you invaluable feedback. ✅ Monitor real-world failures ✅ Feed use cases & data into your test set ✅ Continuously improve Your users will guide you toward what matters most.

6/ Step 5: Know when to stop Unlike products, models may hit a point where further iteration adds no value. Before each new version, ask: Is it worth it? Sometimes, "done" is the right call.

🚀 [NEW POST] The Minimum Viable Model (MVM) Framework The most common mistake I saw in more than 10 years of building AI products? overcomplicating things. You can avoid that with these 5 steps and build AI products that actually work. Here's how 🧵👇

View full thread

3/ Step 2: Build & evaluate a v0 Don’t overthink it—get your first data point. ✅ Create a simple test dataset ✅ Write basic code ✅ Define a metric Plot v0 performance on a graph (X = iterations, Y = performance). It’s all about starting small.

4/ Step 3: Iterate or deploy If v0 works, deploy! (Don’t wait for perfection.) If not, iterate until "good enough." Use your test framework to measure improvement with each version. Avoid overthinking—good enough means ready to ship.

🚀 [NEW POST] The Minimum Viable Model (MVM) Framework The most common mistake I saw in more than 10 years of building AI products? overcomplicating things. You can avoid that with these 5 steps and build AI products that actually work. Here's how 🧵👇

1/ Why MVM? Too many AI products fail because teams chase perfection instead of value. The Minimum Viable Model framework focuses on launching a simple model, gathering real-world feedback, and improving from there. Here’s the 5-step framework 👇

2/ Step 1: Develop intuition & test it Can AI solve your problem? Is the effort small, medium, or large? Start simple: read posts, watch videos, or run small, no-code experiments (e.g., try a task with an LLM in chat). Test your assumptions before diving in.

🧠 Keeping up with research doesn’t have to be painful: LLMs to the rescue! If you’ve ever tried staying up to date with platforms like arXiv, you know how overwhelming it can be. We built a demo notebook to show how LLMs can simplify this process for you. 👇 More details:

View full thread

At Limai, we’re passionate about making AI accessible to solve everyday challenges. From keeping up with research to scaling your business, AI has the power to transform how you work. Let’s connect to explore how AI can work for you!

🧠 Keeping up with research doesn’t have to be painful: LLMs to the rescue! If you’ve ever tried staying up to date with platforms like arXiv, you know how overwhelming it can be. We built a demo notebook to show how LLMs can simplify this process for you. 👇 More details:

View full thread

How it works: 1️⃣ Parses arXiv RSS feeds for titles, abstracts, and URLs. 2️⃣ Asks an LLM to evaluate relevance, explain why it’s relevant, and summarize the main takeaway. 3️⃣ Emails you ONLY the most relevant results—no more wasted time!

🛠️ This demo is part of our new AI & LLMs Demo Repository, where you can explore practical applications of AI in real-world workflows. 📩 Want to streamline your research process? Check out the link at the end of the thread.

🧠 Keeping up with research doesn’t have to be painful: LLMs to the rescue! If you’ve ever tried staying up to date with platforms like arXiv, you know how overwhelming it can be. We built a demo notebook to show how LLMs can simplify this process for you. 👇 More details:

What the demo does: 🔍 Monitors arXiv RSS feeds for new papers. 🤖 Uses an LLM to assess if a paper is relevant to your field of study. 📬 Emails you a summary, key takeaways, and a link to any relevant papers.

You are up to date.