Jan

Jan

@janschulte.com

indiehacker, solopreneur, deep learning building: https://searchattention.com (launching soon!) consulting: https://betalyra.pt

214 followers240 following38 posts

Jan
@janschulte.com

Integrated AI into your SaaS but it feels slow? 11 tricks to make it fast: → Use small models, route up when needed → Minimise output tokens — they're 2-4x more expensive → Stream everything, including tool call results → Design use-case tools, not CRUD wrappers

Jan
@janschulte.com

→ Structure prompts for caching (static top, dynamic bottom) → Disable thinking — it's on by default, adds seconds → Compact conversation history every turn → Strip your system prompt to the bone → Push everything except the response to background queues

Jan
@janschulte.com

Instead of a useless AI emoji generator I wish #apple had used AI to create intelligent printing that doesn't print the whole page black or print in portrait mode even though you selected landscape

Jan
@janschulte.com

This is huge! Perplexity just released their search index as an api and with it a blog article giving some insights into how they rank. According to their announcement the search api is the exact same as the one powering the perplexity application.

View full thread
Jan
@janschulte.com

The gist is: - they start with semantic embeddings and lexical retrieval (probably BM25 and similar) for a first very fast retrieval - then they apply heuristics based filtering and base filters for things like stale content - then using cross-encoders aka rerankers to rank the final result sets

Jan
@janschulte.com

So pretty similar to a typical RAG pipeline but on a giant scale. The search api announcement: www.perplexity.ai/hub/blog/int... The insights article research.perplexity.ai/articles/arc... hashtag#aio hashtag#aiso hashtag#geo hashtag#ai hashtag#llm hashtag#aeo

research.perplexity.aiArchitecting and Evaluating an AI-First Search APIBuilding a scalable Search API that handles 200 million daily queries using hybrid retrieval and intelligent context curation for AI models
Jan
@janschulte.com

This is huge! Perplexity just released their search index as an api and with it a blog article giving some insights into how they rank. According to their announcement the search api is the exact same as the one powering the perplexity application.

Jan
@janschulte.com

if true this would make it an amazing api for AIO research. They also published an article giving some super interesting insights into how their index is constructed.

Jan
@janschulte.com

This feels like traditional banking vs. modern banking like revolut. Google feels like traditional banks that rely on some batch COBOL jobs that some dude in the 70s wrote and that run once a night and if it fails, will run again the night after. also not running on the weekends.

Jan
@janschulte.com

just give me real-time indexing. It's 2025!! The technology is there. Perplexity & co can leverage this to appear always up to date in comparison to Google, Bing & Co

Jan reposted

JP Ahonen 🇪🇺
@jpahonen.eurosky.social

If the United States is willing to extort with Starlink, assume AWS, Google, and Azure clouds are next. Ukraine today, whose turn is it tomorrow? Any US technology is now a liability. We need European solutions as fast as possible.

Toomas Hendrik Ilves@ilvestoomas.bsky.social

If true then the US is now a genuine Russia-style mafia state. Russia extorts with energy-, the US with tech-dependence. I trust the conclusions we draw from this will also turn into a semblance of policy.

Jan reposted

Richard Branson
@richardbranson.bsky.social

President Zelenskyy has been one of the greatest leaders of our time, guiding Ukraine through the darkest period of its recent history, rising up to Russia’s unwarranted aggression, being an inspiration to the Ukrainian people and ensuring that Ukraine’s voice is heard on the world stage.

Jan
@janschulte.com

today I learnt the mime type of xlsx. Is it application/xslx? application/microsoft.xlsx? No! It's obviously application/vnd.openxmlformats-officedocument.spreadsheetml.sheet

Jan
@janschulte.com

The hermes-3-llama-3.1-8b fine-tune by @nousresearch.com is really mind-blowing. Doing some data analysis on json data with it and it returns incredibly consistent & accurate answers. And all this at a ridiculous 4.92GB in 4-bit on my M1. 🙇 #ai #llm #llama

You are up to date.