Reviving CGI: Serving 200M Requests Daily
Description
In this episode of Tech Time Talk, we explore the unexpected revival of CGI (Common Gateway Interface) with expert Jake Gold. Once considered outdated, CGI is making a comeback thanks to modern programming languages like Go and Rust, which enable rapid processing. Jake shares his experience running a CGI program on a 16-thread AMD 3700X, serving over 2400 requests per second—equating to more than 200 million requests a day! Discover how advancements in hardware and software have transformed CGI from a performance bottleneck into a viable web development solution. Join us for an insightful discussion that challenges the notion of old technology being obsolete and inspires innovative thinking in web development.
Show Notes
## Key Takeaways
1. CGI technology is seeing a revival due to advancements in programming languages and hardware.
2. Jake Gold's project demonstrates that CGI can handle over 2400 requests per second with modern setups.
## Topics Discussed
- The basics of CGI and its historical context.
- The performance capabilities of modern CGI using Go and SQLite.
- The impact of hardware advancements on web technologies.
Topics
Transcript
Host
Welcome back to Tech Time Talk! Today, we're diving into a fascinating topic: the revival of CGI, or Common Gateway Interface, in web development. Our expert today is Jake Gold, who’s been testing how well this 90s technology works with modern hardware. Jake, thanks for joining us!
Expert
Thanks for having me! It’s great to be here and talk about something that might seem outdated but is surprisingly relevant.
Host
Absolutely! Now, CGI has been around since the late 90s. Can you give our listeners a quick rundown on how CGI works?
Expert
Sure! CGI essentially processes web requests by starting a new process for each incoming request. It executes the program, then terminates it after delivering the response. Back in the day, this was a bit of a performance killer, which is why languages like PHP and FastCGI became popular.
Host
Right! That makes sense. But you’ve been experimenting with CGI again using Go and SQLite on some modern hardware. What was your motivation behind that?
Expert
Well, after years of thinking CGI was a bad idea, I discovered that technology has evolved. I built a project called datasette-ripgrep that leverages fast subprocesses and it worked incredibly well. I realized that modern programming languages like Go and Rust have such fast startup times that CGI could actually be effective now.
Host
That’s really interesting! So, what did your tests reveal about the performance of CGI with this modern setup?
Expert
With my setup on a 16-thread AMD 3700X, I was able to serve over 2400 requests per second, which adds up to more than 200 million requests a day! That’s a huge number for something many thought was obsolete.
Host
Wow! That’s impressive. And you mentioned that the evolution of hardware plays a role too, right?
Expert
Exactly! Today’s servers come with far more CPU threads, and they’re much faster. A simple virtual machine can have 16 CPUs, which allows CGI processes, running separately, to effectively utilize all those cores.
Host
So, are we suggesting we should start building web applications like it’s 1998 again, but with modern languages?
Expert
In a way, yes! But I wouldn’t recommend it for everyone. It’s more about understanding how things have changed. What used to be a bad idea might not be so bad today due to advancements in technology.
Host
That’s a refreshing perspective! It’s amazing how old concepts can adapt to new contexts. Thanks for sharing your insights, Jake.
Expert
My pleasure! It’s a fun topic and I hope it inspires some people to think outside the box.
Host
For sure! Thanks to our listeners for tuning in. Until next time, keep exploring the tech world!
Create Your Own Podcast Library
Sign up to save articles and build your personalized podcast feed.