Manufacturing loads of tool scripts with ChatGPT

Did you notice how ridiculously easy it is to create small scripts and tools for productivity since ChatGPT-4 is around? Need a script to feed a prompt to ChatGPT from the command line? Boom! There it is. Need a script that suggests a bash commandline suitable for your system with ChatGPT? Boom! Need a script as external tool in IntelliJ that creates a basic unittest for a class? Boom! Got too many scripts and need a script that tells you which of your scripts might be useful for your task? Boom!

The basic prompt

Of course that needs a little startup, and is much about the right prompts. To create a script, I've got the following basic prompt that starts my requests to create scripts.

Please generate a script for the command line for the following task, runnable on a MacBook Pro with Apple M1 Max, that is, arm64 architecture, and MacOS Ventura 13.3.1 . It can be either a bash script using any of the normally present MacOS command line tools or what is installable with homebrew, or a NodeJS script for version 19.5.0 . If NodeJS, it should not require installing any additional libraries. If called without arguments or with the argument --help, the script should print a short description what it does, and what arguments it expects.

The chatgpt tool

Now, if you want something more difficult like that script to use ChatGPT from the command line, that'll need some work, since you want to type out a sufficiently precise specification for ChatGPT to create it. Usually you'll feed it to ChatGPT, inspect the result, edit the prompt and feed it again, until it looks right. Well, there was one bug and I improved some small things, but that was probably still faster than doing it myself. But that's the first one, and the first time is always more work. Here's the prompt, added after the basic prompt. You can skip reading that and just use my script (or rather ChatGPT's, that is) - that prompt just here to give you an idea how a semi-serious specification for a small tool looks like. (If you like, you can see the result of this and the other scripts I'm talking in this blog at GitHub with only some small fixes and changes. Of course I am changing and extending stuff all the time - when you're reading this there will probably be more stuff in the script directory of my chatGPTtools project.

The name of the script is chatgpt. It should submit a single message to ChatGPT and print the answer to stdout. It should be done using NodeJS using fetch.

In addition to option --help, it should have the following command line arguments: - if it gets a single dash, that is a - , it should read the prompt from stdin. Otherwise it should take the command line except the options as prompt to send to ChatGPT. - if it gets -m modelname it should use modelname as a model attribute in the request, instead of the default gpt-3.5-turbo - if it gets -t number it should transmit the given number as 'max_tokens' in the request.

It should access the ChatGPT chat completions API at https://api.openai.com/v1/chat/completions . The API key should be read from file ~/.openaiapi . The request looks like: {"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"This is the promt to submit to ChatGPT"}]}

A successful response is like { 'choices': [ { 'message': { 'role': 'assistant', 'content': 'This is the response to extract and print to stdout'}, 'finish_reason': 'stop', } ] }

If the finish_reason is something other than stop, then print an error message to stderr and abort with exit code 1 . If the request is not successful and has status code 429 then look for a String like "Please try again in 20s." (the number 20 might vary) and wait for that many seconds and then retry, at most 5 times. If the status code is other than 429 for an unsuccessful response, abort with exit code 3 .

createJavaUnittest

How about making a script that passes a class to ChatGPT and has it create an unit test for it and automatically writes that result into a new test file? Here's the prompt, in addition to the basic prompt:

The name of the script is createJavaUnittest. In addition to option --help, it should take exactly one argument: a Java file for which ChatGPT is to create an unittest. The Java file should be piped to the command 'chatgpt' , which should get the following arguments: chatgpt - "End of (filename). Please create unittests with JUnit 4 using an ErrorCollector rule named ec and ec.checkThat; you can use Mockito if mocking is needed. Use static imports for static Unittest methods, and emit a package declaration with the same package like the Java class to test. Do only print the Java code for the test, absolutely no additional text."

Before executing that, the file should be checked for being an non-empty file with extension .java, and the file's absolute path should be checked to contain src/main/java . If not, the script should print an error message. If it contains src/main/java, the output of the chatgpt command should be written to that filename with src/main/java replaced by src/test/java and .java replaced by Test.java . If that directory does not exist, please create it.

If you use my script You will likely want to fiddle with the prompt and give it more instructions on how to write the test. Just try it out and adapt it. You might want to include comments about what to test (only public methods?), testing and assertion style, whether to use an ErrorCollector, what to create assertions for, whether to use Mockito or Hamcrest. A problem I'm not quite sure how to deal with is that it might be sensible to automatically give ChatGPT implemented service interfaces which are implemented by the class to test, as these usually contain javadoc to draw from. And of course it will get troublesome if the class is too long, but that's bad programming style, anyway, right? ;-)

If you're using IntelliJ: a nice way to use it is to create an "external tool" configuration in the IntelliJ settings with createJavaUnittest as program and "$FilePath$" as parameter. While you're at that, you could also add the chatgpt script itself as external tool into IntelliJ and give -f "$FilePath$" "$Prompt$"as arguments. That will call ChatGPT with the currently selected file and IntelliJ will prompt you so that you can enter something for ChatGPT to do. Be sure to select "Open console for tool output", as this will not create any files. For instance, I just selected a class as example and then gave the prompt "Create class PagesPluginService that is a service and has a list of PagesPlugin as an attribute. You can use the given SlingDashboardManager as example for how to handle the pages plugins."

The "search tool" script

After a while you'll probably get so many tools that you don't remember them all from the top of your head. How about a script searching for a tool?

The name of the script is chatgptsearchscript. If the option --help wasn't given, it should collect all command line arguments into the variable prompt. Then it should take the file _usages.txt in the directory containing the script, pipe that into the program chatgpt with the arguments chatgpt - $additionalprompt $prompt where additionalprompt is as follows: "These are the usage descriptions of several command line utilities. Please tell me which script I could use to perform the following task:"

Having ChatGPT write scripts without understanding them?

Of course, you have to at least be able to read and understand the generated code, or you are still lost. OK, not entirely - for fun I tried having ChatGPT write a webpage summarizing bookmarklet without coding a single line myself. That does work, but becomes an annoying lengthy process where you try out ChatGPT's fixes and extensions, describe the error or problems that remain and again apply the new fixes. Not much fun, trust me. But it means even a programming novice could - in theory - have it write scripts for him, though I wouldn't quite know what a novice would do with them.

Extending the scripts

Of course you'll be extending those scripts over time. I did that with e.g. chatgpt, which is rapidly becoming the heart of my ChatGPT toolbox, with a couple of new options. That was where ChatGPT's brother GitHub Copilot cames into play. I went to the print usage string in the script, added the new options and their description. Thus, the Copilot knew what I want without me giving any instructions. The rest was just going into the places where the parts of the new functionality had to be, wait a second, accept the Copilot suggestion and sometimes edit it a little bit. And in one case it needed a bit of discussion with ChatGPT until I found the right idea for the functionality, thinking of how to separate that into several small functions, write the comments for them and have Copilot do it's work. That was a massive time save - normally I'd have thought whether I really wanted to implement that.

Wrapup

The consequence of this is that at the moment my set of development related command line tools, IntelliJ external tools and even bookmarklets (since I'm doing some frontendy stuff at the moment) is practically exploding. I normally have that guideline that if you're doing something 5 times it's worth automating it. That basically changed to "if you expect doing it more than 10 times", automate it. Even with ChatGPT that means that it'll take the first time more time than if I did it by hand, but the experience in scripting, ChatGPT and automating things will likely pay off very handsomely.

So, what's next? The chatgpt-create-a-script script? The DWIM (do what I mean) script? :-) But OK, let's stop for today.