ZDNET’s key takeaways
- ChatGPT Codex wrote code and saved me time.
- It additionally created a critical bug, nevertheless it was capable of get well.
- Codex continues to be primarily based on the GPT-4 LLM structure.
Properly, vibe coding this isn’t. I discovered the expertise to be sluggish, cumbersome, demanding, and incomplete. However it all labored out in the long run.
ChatGPT Codex is ChatGPT’s agentic software devoted to code writing and modification. It could actually entry your GitHub repository, make modifications, and problem pull requests. You possibly can then overview the outcomes and resolve whether or not or to not incorporate them.
Additionally:Â How to move your codebase into GitHub for analysis by ChatGPT Deep Research – and why you should
My main growth mission is a PHP and JavaScript-based WordPress plugin for website safety. There is a most important plugin obtainable without spending a dime, and a few add-on plugins that improve the capabilities of the core plugin. My personal growth repo accommodates all of this, in addition to some upkeep plugins I depend on for consumer help.
This repo accommodates 431 information. That is the primary time I’ve tried to get an AI to work throughout my total ecosystem of plugins in a personal repository. I beforehand used Jules to add a feature to the core plugin, however as a result of it solely had entry to the core plugin’s open supply repository, it could not have in mind your complete ecosystem of merchandise.
Earlier final week, I made a decision to present ChatGPT Codex a run at my code. Then this occurred.
GPT-5 launched
On Thursday, GPT-5 slammed into the AI world like a freight practice. Initially, OpenAI tried to power everybody to make use of the brand new mannequin. Subsequently, they added legacy mannequin help when a lot of their prospects went ballistic.
I ran GPT-5 towards my set of programming tests, and it failed half of them. So, I used to be notably interested by whether or not Codex nonetheless supported the GPT-4 structure or would power builders into GPT-5.
Nevertheless, after I queried Codex 5 days after GPT-5 launched, the AI responded that it was nonetheless primarily based on “OpenAl’s GPT-4 structure.”
I took two issues from that:Â
- OpenAI is not prepared to maneuver Codex coding to GPT-5 (which, recall, failed half my assessments).
- The outcomes, conclusions, and screenshots I took of my Codex assessments are nonetheless legitimate, since Codex continues to be primarily based on GPT-4.
With that, right here is the results of my still-very-much-not-GPT-5 take a look at ChatGPT Codex.
Getting began
My first step was asking ChatGPT Codex to look at the codebase. I used the Ask mode of Codex, which does evaluation, however does not really change any code.
I hoped for one thing as deep and complete because the one I acquired from ChatGPT Deep Research a few months ago, however as an alternative, I acquired a a lot much less full evaluation.
I discovered a more practical method was to ask Codex to do a fast safety audit and let me know if there have been any points. Here is how I prompted it.
Establish any critical safety issues. Ignore plugins Anybody With Hyperlink, License Fixer, and Settings Nuker. Anybody With Hyperlink is within the very early phases of coding, and isn’t prepared for code overview. License Fixer and Settings Nuker are specialty plugins that don’t want a safety audit.
Codex recognized three most important areas for enchancment.
All three areas had been legitimate, though I’m not ready to change the serialization knowledge construction at the moment, as a result of I am saving that for an entire preferences overhaul. The $_POST grievance is managed, however with a distinct method than Codex seen.
Additionally: The best AI for coding in 2025 (and what not to use)
The third space — the nonce and cross-site request forgery (CSRF) danger — was one thing price altering straight away. Whereas entry to the consumer interface for the plugin is assumed to be decided by login function, the plugins themselves do not explicitly examine that the particular person submitting the plugin settings for motion is allowed to take action.
That is what I made a decision to ask Codex to repair.
Fixing the code
Subsequent up, I instructed Codex to make fixes within the code. I modified the setting from Ask mode to Code mode so the AI would really try modifications. As with ChatGPT Agent, Codex spins up a digital terminal to do a few of its work.
When the method accomplished, Codex confirmed a diff (the distinction between unique and to-be-modified code).
I used to be heartened to see that the modifications had been fairly surgical. Codex did not attempt to rewrite massive sections of the plugin; it simply modified the small areas that wanted enchancment.
In a couple of areas, it dug in and altered a couple of extra strains, however these modifications had been nonetheless fairly particular to the unique immediate.
At one level, I used to be curious to know why it added a brand new foreach loop to iterate over an array, so I requested.
As you’ll be able to see above, I received again a reasonably clear response on its reasoning. It made sense, so I moved on, persevering with to overview Codex’s proposed modifications.
All advised, Codex proposed making modifications to 9 separate information. As soon as I used to be glad with the modifications, I clicked Create PR. That creates a pull request, which is how any GitHub consumer suggests modifications to a codebase. As soon as the PR is created, the mission proprietor (me, on this case) has the choice to approve these modifications, which provides them into the precise code.
It is a good mechanism, and Codex does a clear job of working inside GitHub’s surroundings.
As soon as I used to be satisfied the modifications had been good, I merged Codex’s work again into the principle codebase.
Houston, now we have an issue
I introduced the modifications down from GitHub to my check machine and tried to run the now-modified plugin. Watch for it…
Yeah. That is not what’s imagined to occur. To be honest, I’ve generated my very own share of error screens identical to that, so I am unable to actually get indignant on the AI.
As a substitute, I took a screenshot of the error and handed it to Codex, together with a immediate telling Codex, “Selective Content material plugin now fails after making modifications you instructed. Listed below are the errors.”
It took the AI three minutes to counsel a repair, which it offered to me in a brand new diff.
I merged that develop into the codebase, as soon as once more introduced it right down to my check server, and it labored. Disaster averted.
No vibe, no movement
Once I’m not in a rush and I’ve the time, coding can present a really nice way of thinking. I get right into a kind of movement with the language, the machine, and what looks like a connection between my fingers and the pc’s CPU. Not solely is it a variety of enjoyable, nevertheless it will also be emotionally transcendent.
Working with ChatGPT Codex was not enjoyable. It wasn’t hateful. It simply wasn’t enjoyable. It felt extra like exchanging emails with a very recalcitrant contractor than having a gathering of the minds with a coding buddy.
Additionally: How to use GPT-5 in VS Code with GitHub Copilot
Codex offered its responses in about 10 or quarter-hour, whereas the identical code would most likely have taken me a couple of hours.
Would I’ve created the identical bug as Codex? Most likely not. As a part of the method of considering by way of that algorithm, I probably would have averted the error Codex made. However I undoubtedly would have created a couple of extra bugs primarily based on mistyping or syntax errors.
To be honest, had I launched the identical bug as Codex did, it might have taken me significantly longer than three minutes to search out and repair it. Add one other hour or so at the least.
So Codex did the job, however I wasn’t in movement. Usually, after I code and I am inside a selected file or subsystem, I do a variety of work in that space. It is like cleansing day. If you happen to’re cleansing one a part of the lavatory, you may as effectively clear all of it.
However Codex clearly works greatest with small, easy directions. Give it one class of change, and work by way of that one change earlier than introducing new components. Like I stated, it does work and it’s a useful gizmo. However utilizing it positively felt like extra of a chore than programming usually does, though it saved me a variety of time.
Additionally: Google’s Jules AI coding agent built a new feature I could actually ship – while I made coffee
I haven’t got tangible check outcomes, however after testing Google’s Jules in Could and ChatGPT’s Codex now, I get the impression that Jules is ready to get a deeper understanding of the code. At this level, I am unable to actually help that assertion with a variety of knowledge; it is simply an impression.
I’ll attempt working one other mission by way of Jules. It is going to be attention-grabbing to see if Codex modifications a lot as soon as OpenAI feels protected sufficient to include GPT-5. Let’s take into account that OpenAI eats its personal pet food with Codex, which means it makes use of Codex to construct its code. They may have seen the identical iffy outcomes I discovered in my assessments. They may be ready till GPT-5 has baked for a bit longer.
Have you ever tried utilizing AI coding instruments like ChatGPT Codex or Google’s Jules in your growth workflow? What sorts of duties did you throw at them? How effectively did they carry out? Did you’re feeling like the method helped you’re employed extra effectively? Did it sluggish you down and take you out of your coding movement?
Do you favor giving your instruments small, surgical jobs, or are you on the lookout for an agent that may deal with big-picture structure and reasoning? Tell us within the feedback under.
You possibly can comply with my day-to-day mission updates on social media. Be sure you subscribe to my weekly update newsletter, and comply with me on Twitter/X at @DavidGewirtz, on Fb at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, on Bluesky at @DavidGewirtz.com, and on YouTube at YouTube.com/DavidGewirtzTV.
