About the role
from job pageImagine if the internet were a database. What would you build? SELECT * FROM Internet; We’re building out the early engineering team of expand.ai, where we’re (a) solving a problem the world urgently needs to be solved, that is (b) one of the most challenging problems in technology, (c) alongside a team of exceptional engineers (d) offering you the chance to have a massive impact, (e) while having fun (f) and be reward accordingly. (a) Problem While LLMs are democratizing intelligence, access to data remains a huge bottleneck. We're changing that by building state-of-the-art web extraction agents that structure millions of websites—one at a time. Scraping is not new. Folks have written XPaths, manual beautiful soup scripts, etc. However, two things changed: Scraping is needed even more than before. Almost any AI application needs access to some data from the internet. We’re in the post-LLM era. While LLMs are too expensive to run over the internet, they enable previously impossible capabilities. That’s why we’re on a mission to build a reliable data layer for the web. When we soft launched in September, we got an insane amount of interest, with over 170 demos booked in 24 hours. This confirms how much of a problem this is for folks. We’re luckily not alone on this mission and have backing from the best: YCombinator, Guillermo Rauch (CEO, Vercel), Swyx (Founder, smol ai), Sarah Guo (Conviction Embed), Alana Goyal (basecase capital), Max Claussen (system.one), Ellen Chisa, Charly Poly, … and many more! (b) Tech challenges I (Tim) have been coding for 15 years and never faced this difficulty. We’re building a highly dynamic system that needs to deal with the undeterministic nature of the internet, at scale. Some of the challenges we’re tackling: Building a fair system across multiple tenants (noisy neighbors)
