juan M

I actually had the final push to create the blog when trying to answer a former colleague’s post with chatGPT. But indeed, I’ve been ruminating on many things “chatGPT” in the past weeks wanting to write about it. Not that I have made a full mental model about it (just yet?), but I do have ideas and I often find takes which either miss the mark or are based on some misunderstanding.

In this case, my former colleague Martin Pergler wrote something on LinkedIn (not totally sure if the link will show ok to everyone clickly) to which I mostly agree with, but wanted to provide some additional points that I thought would extend on this view. Unfortunately for my intentions (probably good for other ocassions!) LinkedIn limits the number of characters in a comment to a post, so I actually sent him my reply via DM. I didn’t feel like making a full post in my own LinkedIn as I preferred to have a “small forum” discussion, still in public, but for different reasons I didn’t want to have that on LinkedIn (will write something about LI in some point). This was actually the final nail that made it clear to me that I should have the blog.

So, the text of this post was (linking it again):

#chatgpt . Impressive to be sure, but colour me less enthralled (or scared) than some.

I think it has great potential as smart search++, quickly summarizing and slightly synthesizing (or at least structuring) a more-or-less state-of-the-art answer to custom questions, though currently with the limitations of its knowledge set ending in 2021, and sources not provided.

But it is not intelligent, in the colloquial not formal sense. Its answers (I’ve tried a number of queries in my field of #riskmanagement, #decisionmaking under #uncertainty) summarize but don’t transcend ambient generally-accepted wisdom, and embed its flaws and biases.

Maybe that’s precisely its value/transformative ability: Decrease the effort to get to a C+ grade answer to mildly complex questions. Google search and Wikipedia have solved the problem of being completely uninformed about a topic. ChatGPT helps put basic answers to a question (vs topic) at one’s fingertips. That’s amazing, will be super helpful, but I’m not yet convinced earthshattering in a way that would save the planet, solve social issues—or sunset careers for that matter.

As a #mckinsey alum, the closest analogue I see is early 2000s McKinsey R&I (research and information) staff, who were super useful in providing a reasonably structured, primer-level answer to weird questions you would ask on a project (often more, if you got a true expert; I’m referring to first-line). But whose role specialized and in its original form became less relevant as Internet search became so much better and more ubiquitous.

(My mind isn’t made up regarding the “generative” part, in particular the decent-prose answers. A great parlor-trick to be sure, but will it achieve anything more than choke information channels with superficially well-written but low baud-rate content? I may be overly critical, as someone who writes fairly easily and is picky with my words.)

source: https://www.linkedin.com/feed/update/urn:li:activity:7028289540284964864/

And so my answer to him is that I treat the current model of chatGPT and related variants of Language Learning Models (LLMs) akin to being an assistant of limited cogent ability, but with super poweres (vs. a human) to absorbe enormous amounts of previously written content, and then provide a sort of statistical summary of that content (and also a statistical summary on how to write about that content). One key thing about that formulation of a summary is that depending how weights are assigned, the output can be very different (and often wrong for your particular need, or plain wrong in terms of truthiness). Then, as any other tool, by being aware of the limitations we can find the best use cases, and for each use case the proper way of using the tool (for instance taking into account the cost of being wrong vs. the time and effort spent in a given task).

On one hand, the “AI” field has been around for decades, so you could argue that the time until the next step change to get something with an order of magnitude of more interesting and value added use cases may be several years away. Others may say that actually chatGPT is the first of its kind (and they would be right in the narrow sense of wiring a chat interface, with an LLM as backend with some sanitising logic on the input and output), and that they are only going to become better very quickly (more training data, smarter ways of applying the weights, faster/cheaper ways of retraining, etc. etc.). LLMs in their current form (strongly based on certain common neural network architectures such us Transformers) may have structural limitations that may put a boundary their ability to increase their “reasoning” ability, but we may as well see improvements of those architectures (or totally new ones, and possibly, combinations of them).

I am quite bullish on the technology, in terms of its application in education (higher education, professional education, etc), health, and many other industries as part of the general “software is eating the world” narrative from A16Z (https://a16z.com/2011/08/20/why-software-is-eating-the-world/), but I am far from certain on the speed of this actually showing up in the social wellbeing and its related statistics. Also, I can see how a direct use case can be to enhance productivity of knowledge workers (again, with some understanding on the limitations). Of course, my positivity is likely influenced by the fact that my version of statistical averaging of opinions on this matter is highly skewed by entrepreneurs and ML/AI scientists which I’ve been following (and been following a lot in the past 3 months).

I wanted to give a pass of this text through chat GPT for conciseness and remove likely language kinks from a non-native speaker, but chatGPT is currently at capacity and I decided it was not worth the wait (may write a revised version later hehe).