Episodes from Alexmorleyfinch

Build a 500k Domain Search Engine in a Weekend

In this episode of Tech Talk, we dive into the inspiring story of Marlin, who built a personal search engine for makers featuring over 500,000 domains, all in just a weekend for under $10. Frustrated with existing search tools drowning in corporate content, Marlin set out to create a platform that highlights creativity in art, code, and poetry. He shares his unique approach to web crawling, focusing on key metadata instead of full HTML pages, and discusses the technical components involved, including a fetcher, a local model for summarization, and a management steward for domain activity. Learn about the challenges he faced, his strategies for keeping costs low, and how his personal search engine aims to provide a curated experience for creators. Tune in for insights into software development, web crawling, and the importance of personal projects in tech.