A lot of people get prickly when asked about AI. They are upset, I think unreasonably so, when AI models get something wrong.
They say:
Or:
I think they kind of have a point. Even state-of-the-art models hallucinate and can set you down the wrong path.
But this is shortsighted. These detractors aren't going deep enough in their reasoning. Yes, models hallucinate. Yes, they get it wrong. You could even say models get it wrong a lot.
I'll try to show you why it's okay that models get it wrong a lot. I'll also try to show you how to use them to your advantage in spite of this.
This website is a quiz website. I have a simple rust binary that hits Hacker News' official API to scrape the front page every day and insert the links into postgres. Then, I have o3 generate a multiple choice question for the top link for each day of the past week. Finally, I have a simple front end that renders the questions, lets you take a quiz, and share your results! I also collect quiz takers answers so you can see how you scored relative to others and how many people selected each multiple choice option.
AI detractors take major issue with me using o3 to generate the quiz questions. And let me tell you, they are completely correct when they claim the generated questions are wrong a lot of the time. In fact, in my experience, the generated questions are stupid or factually incorrect about 40% of the time.
But again, that's okay. We're humans. We can still use AI to our advantage. We can still be more productive with it.
How? I simply asked o3 itself to make the binary that generates questions interactive. It lets me review each question and I can tell it when a question is stupid and I can tell it WHY a question is stupid before it tries to regenerate a new question.
That's the gold right there. That's the rub.
Each week, I run one binary. Each week, I read through 10 questions it generated. Each week, I have to do about two to three back-and-forths with about three or four of the questions.
And that's it!
I know these hallucinations are so infuriating that /r/programming refuses to even entertain the idea that AI models are a net positive by any metric that matters.
But, if you get over the initial frustration they cause, you can find a method of interaction that tremendously boosts your productivity.
It takes me less than 10 minutes to create the quiz questions each week, now. It would've taken me easily two to three times that much time had I not had access to AI. And that says nothing of the time saved developing this website, too.
(If you want play the quiz, you can do that here.)