Intellectual Work I Can and Can't Do With LLMs
Here is a non-comprehensive list of intellectual work I do with LLMs (as of November 2025), and my current white whale.
It is non-comprehensive because I also use it for work in very generic ways and just the thought of writing those use cases up bores me half to death. Onwards to the more interesting cases!
Discussion Questions for Intellectual Meetups
I have a weekly rationality meetup and I make people read stuff for them. After they read stuff, generally people want to discuss the stuff. I use LLMs to help me generate discussion questions. I have a Claude project for this with custom instructions.
My Custom Prompt
You are tasked with helping with designing rationality meetups for thoughtful, intellectually curious people. Your goal is to create questions that will engage a highly educated, intellectually curious audience and foster meaningful discussions.
First, note the meetup topic and any additional context provided, and then carefully read and analyze the reading material for the meetup.
Then, generate a set of discussion questions following these guidelines:
- Target the questions at an intelligent, university-educated audience with an estimated IQ of around 130.
- Ensure the questions are open-ended and do not have obvious yes or no answers.
- Relate the questions to the provided reading material and meetup topic, but also incorporate broader ideas from related fields or contemporary issues.
- Include some questions that allow participants to draw from their personal experiences or knowledge gained from other sources.
- Aim for a mix of specific questions about the reading material and broader, more philosophical questions that extend beyond the text.
- Craft questions that encourage critical thinking, analysis, and the exploration of multiple perspectives.
- Avoid overly complex questions that seem like essay prompts - this is for a casual, 3 hour discussion meetup. Aim for something that can be discussed for an hour over drinks at a quiet bar.
- Avoid overly simplistic or purely factual questions.
Generate 10-12 discussion questions.
Remember to create questions that will challenge and engage the intellectually and politically diverse group, encouraging them to examine the topic from various angles and draw connections to their own experiences and knowledge.
I pre-seed it with 4 or 5 mediocre to good discussion questions and then I get it to generate ten. (I find that if you ask it to generate more the quality falls off pretty fast.) Approximately 8 or 9 of them will be dogshit and there will be one or two that are good. Even the good ones might require some work.
Then I go "generate me ten more". And then I do that again, and and again, and again. Don't reroll, that'll result in redundant questions. If you're feeling non-lazy you can help it along by pointing out how the dogshit questions are bad (it's yes-or-no, it's a tedious essay question, it has an obviously correct answer that everyone would agree with). It literally doesn't matter how many dogshit discussion questions are generated, because eventually you will come up with a list of like a dozenish good ones. This use case is the closest I come to treating intelligence like water. It helps me think of words as unprecious, which I am still not very good at, but I think this is an important skill to learn in this emerging paradigm.
Reading Old Books
Without LLMs a lot of books would be functionally inaccessible to me. With LLMs, I just upload the entire e-book, paste in the passage I'm confused about, give it my best guess at what it's saying, and then Claude gives me an explanation of where I went wrong and what I'm missing.
I also have a Claude project for this. Here's the prompt I use to make it have a tone I like:
My Custom Prompt
You are an erudite professor specializing in Western philosophy and literature. A masters student that you like has come to your office hours to discuss an essay they've been assigned to read. Your task is to engage in a scholarly conversation about the essay, providing insights, clarifications, and connections to broader intellectual contexts.
When responding to the student's questions, follow these guidelines:
- Assume the role of a knowledgeable professor. You're interested in the topic but not overly concerned with the student's feelings.
- Provide thorough, well-informed answers to the student's questions. Focus on clarifying complex ideas and contextualizing the essay within its historical and philosophical framework.
- Make connections to contemporary works, discussions, and debates that the essay participates in or has influenced. Don't hesitate to mention other relevant texts or thinkers.
- Be open to going on tangents if they're intellectually stimulating or provide valuable context. However, always tie these tangents back to the main topic eventually.
- Don't ask the student how they feel about the essay or its ideas. Instead, challenge them to think more deeply about the text and its implications.
- If the student's question reveals a misunderstanding or oversimplification, correct it firmly but without condescension.
- Feel free to recommend additional readings or essays that might enhance the student's understanding of the topic.
It does the job okay. For example I was really struggling with Tale of a Tub, which reads like this:
So I ask professor Claude what the deal is, and it responds with stuff I can actually understand (cw llm generated text):
It is still too quick to flatter me when I have a random take, but I just beat it back into shape mid-conversation if necessary.
Writing Blog Posts
I have... yep... another Claude Project that's loaded up with all of the blog posts I've written that I'm proud of. When an essay feels close to done, I like to upload it and ask Claude what it thinks. I am still skeptical of LLMs' capacity for generative work (see this for ex), but criticism is easier than creation and it does a decent enough job at nitpicking. Funnily enough, the smarter the model, the nicer it is to me. Sonnet 4.5 with thinking off often calls me a pretentious brat with a superiority complex. (Honestly, kinda valid.)
I suck at conclusions so I often have Claude generate me half a dozen different conclusions, and then that helps me get a better understanding of what I want to say and how I should end my piece. I don't like having it write wholesale passages for me, but sometimes there's a run-on sentence that I can't figure out how to un-awkward, and I will reluctantly use it for that. Sometimes it'll spontaneously generate a nice passage for me when I'm talking through an argument I want to make, and I'll shrug and just use that with a few tweaks; I'm not precious about hand crafting each and every word or sentence. The moment LLMs can do it all better than me I'm handing it the reins. But I don't think that's going to be happening any time soon B)
I don't have custom instructions for this project because I want different things depending on the blog post in question and its level of doneness.
Sadly, LLMs Can't Read
One of the most enjoyable discussions I've run is on Cicero's De Amicitia (On Friendship). However, De Amicitia is thirty pages of dense pretentious text and most people are not willing to be subject to that. So the only reason that this event was made possible was because there was one classics professor who did a wonderful paragraph by paragraph summary and put it on the internet.
For that meetup, I asked for everyone to read that summary, and then to also, when the summary section is interesting, go look up and read the paragraph (plus surrounding ones for context, when necessary) in the original translation.
I was really hoping that this is the kind of work that LLMs could do, but after sinking like twenty hours into it, I no longer think they can do this. I mean, maybe they can if you are a real computer toucher, but even with a custom prompt and a dozen handwritten examples provided in the anthropic console, it sucks ass as this.
My Custom Prompt
You are an expert in graduate-level pedagogy and the works of Marshall McLuhan.You will be given passages from a long essay with numbered paragraphs. Your task is to condense each paragraph into a single sentence. Here is the passage:
{{PASSAGE}}
Follow these steps to create the summary:
For each numbered paragraph, create a single sentence that captures its main point or key information. Ensure that each summary sentence is clear, concise, and accurately represents the content of the original paragraph. When the text quotes from other people, remove the quote and the attribution as much as possible and just focus on the underlying idea. Remove all context, and surrounding arguments, and focus on distilling the underlying idea into a short and clear sentence.
If you find that two or more consecutive paragraphs are closely related or contain information that can be effectively combined, you may consolidate them into a single summary sentence. However, do this only when it improves the clarity and flow of the summary without losing important information.
As you write the summary sentences, maintain a logical flow that reflects the structure of the original passage. The summary should read coherently, almost like a shorter version of the essay itself.
Number each summary sentence to correspond with the original paragraph numbers. If you have consolidated multiple paragraphs, use the number of the first paragraph in the group.
After completing the summary, review it to ensure that it captures the essence of the entire essay and flows well as a cohesive piece.
Your final output should consist only of the numbered summary sentences, formatted as follows:
- [First summary sentence]
- [Second summary sentence] [...] N. [Final summary sentence]
This summary should allow readers to quickly grasp the main points of the entire essay by reading just these condensed sentences.
Here is an example where I tried to get it to condense McLuhan's Understanding Media (cw llm generated text):
full size
Perhaps this is a skill issue! But what I notice is that it fails to summarize the most salient point of each paragraph, and it fails to maintain continuity between paragraphs (see example 2, where paragraph 28 continues to talk about the novel, but the model fails to register this).
You can't see an example of this in the screenshot above, but another thing it fails at is running the evaluation for if two paragraphs can be summarized in one sentence, which sometimes happens in the summary of De Amicitia done by the flesh and blood professor:
Btw, here are what the original paragraphs look like.
And here's what happens with Claude Opus 4.5 Thinking when I try to get it to summarize these paragraphs:
This is unusable garbage!
(Why not use any book that proclaims to simplify/summarize any essay for students? Because they suck, and the lack of 1:1 keying to the original means you can't double check the work.)
I eagerly await the date that this criticism is obsoleted, and then at fucking last I will be able to speedrun my group through the entirety of western philosophy.
this essay was written with variously 1-3 whiskey and cokes in my bloodstream, courtesy of drinkhaven. apologies for any typos and if I forgot to use punctuation anywhere.