Absolutely! I will certainly be experimenting with the new functions as you suggest.
One thing which I haven't seen discussed elsewhere is the tension between the output format and the underlying task. When I ask GPT to use a simple, natural output format it does *better* at the actual code editing task. If I ask it to output using a more technical format like `diff -c` or heavily structured json formats... it "gets distracted" and does worse at the underlying coding request.
It writes worse code if you ask it to output edits in a terse, machine-readable diff format. It writes better code if you let it show you the code edits in a simple way.
For GPT3.5 this means I have to let it just type the whole source file back to me, with the edits included. GPT-4 is able to output a very simple diff-like format, but struggles with `diff -c`, etc.
So it will be interesting to see how the new function capabilities affect this tradeoff. Perhaps it is now "fluent" in function-json and so won't be "distracted" by that output formatting.
My guess is the new function training might overall slightly take away from the capacity for other types of reasoning, since it's a fixed capacity. But hopefully significantly less impact than including the instructions on the fly, since it's mostly baked in to the model now which should be a more efficient encoding.
One thing which I haven't seen discussed elsewhere is the tension between the output format and the underlying task. When I ask GPT to use a simple, natural output format it does *better* at the actual code editing task. If I ask it to output using a more technical format like `diff -c` or heavily structured json formats... it "gets distracted" and does worse at the underlying coding request.
It writes worse code if you ask it to output edits in a terse, machine-readable diff format. It writes better code if you let it show you the code edits in a simple way.
For GPT3.5 this means I have to let it just type the whole source file back to me, with the edits included. GPT-4 is able to output a very simple diff-like format, but struggles with `diff -c`, etc.
So it will be interesting to see how the new function capabilities affect this tradeoff. Perhaps it is now "fluent" in function-json and so won't be "distracted" by that output formatting.