The GenAI wave is almost 2 years old now, since ChatGPT moved it from nerdy computer science research and into a simple web-based interface that anyone can use. In that 2 years - 2 years that feel like forever, even in this post-COVID timeline - GenAI has been the most-hyped, most-funded, most-misunderstood technology of this generation. The last time there was this much hubbub about a tech was the Dot Com Bubble just after Y2K.

Problem 1: Attention vs. Understanding

I think I understand at a really basic level how LLMs work - they’re basically linear algebra engines applied to huge (and huger) collections of source materials, looking for patterns and connections between tokens - words, parts of words, strings of words. LLMs can’t “understand” the content they’re analyzing - they don’t have to. All they do is map the “attention” that connects tokens. It’s a reductive process - mapping connections between tokens that are fed into the process. Creating a kind of average, generating most-likely sequences of tokens based on what it’s ingested as input during training. That’s it. But, at the scale of cloud computing and literal warehouses of CPUs and GPUs and gigawatts of power, Attention Is All You Need.

But the language that is used to describe GenAI is strongly rooted in how we talk about people and human intelligence - learning, training, thinking, reasoning. And so, we conflate what GenAI is actually doing (mapping attention between tokens) with what we believe intelligence is (understanding, making meaning, building upon concepts, creating new ideas).

And there’s the first real conflict - GenAI should be thought of as being different than “real” intelligence. But, the size of the LLMs, combined with the natural language interfaces for interacting with them, means we apply our natural human instinct for pareidolia and hey presto, it’s “intelligent” (even though it isn’t).

BUT. How do we really know how our own brains “know” and “think”? We have models, based on whatever technology is important at the time. The brain is like a steam engine. It’s like a series of tubes. It’s like a computer, with storage and processing. It’s like a neural network that maps connections between fragments of experiences. And, when quantum computing takes off, we’ll start describing the brain as a quantum process. Etc. So, not knowing how LLMs “think” or “reason” or generate specific responses to prompts isn’t that different from us not being able to provide a stack trace for our own thoughts. At a high level, it’s all black boxes.

Problem 2: Insatiable hunger for resources

So, we have sprawling datacentres full of energy-hungry CPUs and GPUs and storage and networks, crunching any and all content that people can feed into them, spitting out mappings of tokens and the connections between them. And the hunger is insatiable - Sam Altman is trying to convince governments to build multiple 5GW power plants to feed datacentres, while also trying to convince hardware companies to increase production capacity by ridiculous multiples.

If we get past LLMs not “thinking” the way we think of thinking, and see them as something useful, we still have to reconcile the environmental impact of these things and the vast resources and money that they burn.

This is the second conflict. If we were to build multiple 5GW power stations, we could power entire large cities and reduce their carbon footprints. Instead, we’re marching deeper into climate crisis as we actually consider building these things to power the products of billion-and-trillion-dollar companies.

We don’t tend to talk about the resource demands of other software that we use. Our use of learning management systems (what’s the carbon footprint of Brightspace?), videoconferencing platforms (what’s the energy demand of Zoom?), student information systems (Peoplesoft, Destiny1?), communication (M365?), collaboration (Teams?), etc. And we should absolutely be talking about this as well - especially as every single application we use is building in GenAI tools and their resource demands will only increase. How do we think about reducing our environmental impact, if literally everything we use is increasing its resource demands - likely exponentially?

And we need to pay for all of this. Public university budgets have been facing both effective and real cuts every year for well over a decade. There isn’t a big pile of money to use for this - and if there was, there are other strategic things we should focus on before funding the next big shiny. Every dollar spent on GenAI is a dollar that’s not available to do the things we’ve previously agreed are important.

Problem 3: Welcoming Our Robot Overlords

LLMs are advancing incredibly rapidly - and we’re all witnessing this and interacting with this in realtime. These applications now generate compelling text, images, video, and audio.

The latest mindblowing tech demo is Google’s NotebookLM, and its ability to generate 12-minute-long .WAV audio files containing realistic-sounding “conversations” between two synthetic podcast hosts, discussing whatever document you upload. This software is impressive as hell, but it’s also disconcerting. The 12 minute podcast is shockingly realistic - not long ago, even just the text-to-speech part of this would have felt like science fiction, but the software parses documents, writes a script, and produces a podcast in under 5 minutes.

Other GenAI tools “analyze” data and produce interactive visualizations. They write software that mostly works (and is getting better every week). They create photorealistic images (which have quickly progressed from crude and error-riddled, through obviously-AI-generated shiny surfaces, to shockingly realistic) and audio and video. The tools are being baked into our computer operating systems, into our phones, into the software we use every day. It will be pervasive, and it will be used daily by hundreds of millions of people.

And this is the third conflict - we are outsourcing creativity. Patrick Finn repeats the mantra “you are what you repeatedly do,” and it applies here as well. As we offload parts of our creative processes to GenAI, it becomes a pattern that becomes ingrained. McLuhan’s “we shape our tools and, thereafter, they shape us.” Do we want machines to be this integral to our creative processes? I think this goes beyond “but it’s just a calculator or a spell checker or Photoshop” kind of thing. Calculators don’t do things on their own, even if they’re programmable. Photoshop has enabled artists to manipulate pixels for over 30 years now, but hasn’t - until very recently, when it started baking in GenAI - created images.

Problem 4: I’m part of the problem

I’ve been exploring various LLM tools because I think it’s essential that I’m able to have informed conversations about what LLMs can do, what they can’t do, what they should/shouldn’t do, and what the experience of actually using them is. But, in the process, I’m complicit in normalizing these things, in making GenAI palatable. In pushing the agendas of VC-backed Silicon Valley startups (who are now valued in the hundred-billion-dollar range, so “startup”?). In building the case to divert scarce university resources to pay for and support all of this.

But I can’t just blink 3 times and wish GenAI away. It’s here. Our students and instructors and team members will be using it more and more. Our BoG will demand an institutional stance on GenAI. It will only get “better” (higher quality output) and “worse” (environmental impact, demand for resources).

I honestly don’t know what to do with all of that. I don’t want to be a GenAI evangelist. I don’t want to normalize this, or to prop up business models built around extraction of resources and creativity. If the resource and environmental impact problems were solved, would it suddenly become ethical to use LLMs? If the ethical issues around intellectual property and training of LLMs were solved, would I still be this conflicted? I desperately want a simple answer to all of this, and know that it’s just not possible. This is complicated and messy and problematic.

Update

I fed this post into llama3.1, running on my own laptop, and asked it to provide a summary.

Which is a decent summary, and something that would get the ideas across without having to wade through a 1,400 word blog post.