What I make of ChatGPT

I guess I’ll put my grain to the massive pile of comments regarding Open AI’s new ChatGPT.

Here is an example:

This is a command for a POSIX system. fd is an utility similar to find. Describe what it does

fd -e png . assets/ |
  xargs file |
  cut -d, -f2 |
  tr 'x' '*' |
  bc |
  awk '{x+=$1}END{print(x)}'

The response:

This command uses the fd utility to search for files with the .png extension within the assets/ directory and all of its subdirectories. It then pipes the output of fd to xargs, which is used to run the file command on each of the files found by fd. The output of file is then passed to cut, which is used to extract the second field from each line (delimited by commas).

Next, the output of cut is passed to tr, which is used to replace all occurrences of the letter x with an asterisk (*). The output of tr is then passed to the bc command, which is a command-line calculator that evaluates the mathematical expressions passed to it. In this case, the expressions are the result of replacing the x characters with * symbols in the output of cut.

Finally, the output of bc is passed to awk, which is used to sum up the values and print the total. The awk command uses the x+=$1 expression to add the value of the first field on each line to the x variable. At the end, the END keyword is used to print the value of x, which contains the sum of all the numbers.

In summary, this command searches for all .png files within the assets/ directory and its subdirectories, runs the file command on each file to extract some information, performs some transformations on the output of file, calculates the sum of the resulting values using bc and awk, and finally prints the total.

Hmm, it seems it needs something, let’s do this

The output of file when ran on a .png file is:

assets/character.png: PNG image data, 407 x 408, 8-bit/color RGB, non-interlaced

Please reformulate your description taking this into account

Response:

[…]

The output of file is then passed to cut, which is used to extract the second field from each line (delimited by commas). This field contains information about the dimensions and color depth of the image, such as 407 x 408, 8-bit/color RGB.

[…] This effectively replaces the x character used to separate the dimensions of the image (e.g. 407 x 408) with an asterisk, which is used as the multiplication operator in the bc calculator. For example, the output of tr for the example file output above would be 407 * 408, 8-bit/color RGB.

[…] [Is] then passed to the bc command […] the result of replacing the x characters with * symbols in the output of cut, and they represent the dimensions of the image. For example, 407 * 408 would be calculated by bc to give the total number of pixels in the image.

[…] In summary, […] calculates the total number of pixels in all the images using bc and awk, and finally prints the total.

It’s not exactly accurate, but it gets the idea at the end.

Rust pitfall

Here is a short interaction for code gen

The following function takes a mutable reference to a slice and two indices a and b, It returns two mutable references to elements of slice, the first is the element at index a and the second at index b. Please replace the TODO comment with the actual implementation

fn two_muts_of_slice<T>(slice: &mut [T], a: usize, b: usize) -> (&mut T, &mut T) {
  assert_ne!(a, b);
  // TODO
}

Response:

fn two_muts_of_slice<T>(slice: &mut [T], a: usize, b: usize) -> (&mut T, &mut T) {
  assert_ne!(a, b);
  assert!(a < slice.len() && b < slice.len());
  let (first, second) = slice.split_at_mut(b);
  (&mut first[a], &mut second[0])
}

Ahah! Can you spot the mistake?

Well here is what happens next. I ask:

Is this code correct if a > b?

It responds:

No, the code would not work correctly if a is greater than b. This is because the split_at_mut() method splits the slice at index b and returns a mutable reference to the elements before index b as the first sub-slice, and a mutable reference to the elements at and after index b as the second sub-slice. If a is greater than b, the element at index a would be in the second sub-slice, not the first.

Here is an updated implementation that works correctly for any values of a and b:

fn two_muts_of_slice<T>(slice: &mut [T], a: usize, b: usize) -> (&mut T, &mut T) {
  assert_ne!(a, b);
  assert!(a < slice.len() && b < slice.len());

  if a < b {
    let (first, second) = slice.split_at_mut(b);
    (&mut first[a], &mut second[0])
  } else {
    let (first, second) = slice.split_at_mut(a);
    (&mut second[0], &mut first[b])
  }
}

Also notice how it’s almost gaslighting me, I somehow get the feeling it’s my fault if I didn’t tell it to account for the obvious (for me at least) “edge case”.

The updated code is very close to what I would have written myself, so congratulation!

Other prompts

There is a few more tasks I played with ChatGPT. It’s really good at manipulating code. For example, I got it to refactor duplicate code, or extract multiple printlns into a single constant string. It’s also pretty decent at describing programs, even non-trivial ones. Especially useful for bash scripts I think, but also seemingly obfuscated optimized code. It’s also a nice crutch if you want a starting point from where to write some code (good riddance, writer’s block!).

In general, it’s just excellent when you know exactly what you want and/or have the expertise necessary to nudge it in the right direction. But it’s just awful when you don’t know what pitfall to look for. It’s also downright consistently wrong when asking for guidance or recommendations on specialized topics. It’s output is still useful as a way to navigate a sort of concept tree. It’s much easier to learn about rolling hash interactively through ChatGPT that trying to decipher the Wikipedia article, although ChatGPT has been both consistently and persistently wrong in its applicability. Same with Yocto Linux.

Cyberiadic AI

It reminds me a lot of the various machines Trurl builds in The Cyberiads, as soon as it is finished, it replies to “what four and four does?” with “seven” and persists into defending that point so much so that it starts uprooting itself and chasing its creator.

Of course, the most memorable machine of Trurl’s is the Electronic Bard. A machine so good at poetry that it causes every artist to give up on life.

Note that I’m now kinda in the same boat as the poets in the Cyberiads. What’s the point of writing any code or text if ChatGPT can generate in a second a tome ten times more efficient or elegant? ChatGPT is definitively not capable of that, but this is a question I ask myself every time I write something now, it’s a bit terrifying.

I couldn’t resist reproducing the prompt Klapaucius uses to test the machine and results in the most amazing bit of sci-fi poetry ever written (The English translation is remarkable)

Let’s have a love poem, lyrical, pastoral, and expressed in the language of pure mathematics. Tensor algebra mainly, with a little topology and higher calculus, if need be. But with feeling, you understand, and in the cybernetic spirit.

ChatGPT:

In a world of ones and zeros,
Our love is a complex vector,
A tensor of boundless grace,
Defined by a single equation.

Our passion is a topological knot,
A Moebius strip of endless twists,
Our bound a higher-dimensional space,
Where hearts and minds merge and coexist.

And as we navigate this digital realm,
Our love remains constant,
A matrix of unbreakable trust,
A formula for perfect harmony.

It picked up amazingly the vocabulary in the quoted domains of mathematics! Also quotes a mathematician. The version from Trurl’s machine is of course written by Stanisław Lem (and Michael Kandel for the English version). It captures perfectly the naivety and horniness of love poems and makes fun of it (in the cybernetic spirit, you understand) masterfully.

The three paragraphs ChatGPT generated are nothing compared to it (though make sure you check the rhymes more carefully, there are more than at first glance!) but it’s beyond anything a computer ever was capable of doing previously.

Comparing this to state of the art in 2018 (merely 5 years ago!) we are about five orders of magnitude better.

Of all the sci-fi AIs we conceived in the 20th century, we end up with the AIs from the Cyberaids. Amazing! Looking forward to countering superintelligent entities with bureaucracy.