Kevin Zollman
@kevinzollman.com
Philosophy and Game Theory at Carnegie Mellon 🦚 Research the interface between philosophy, economics, and biology 💱 www.kevinzollman.com
I think this strategy is also more likely to succeed. I remember when faculty used to say "no wikipedia." No one listened to them.
But when faculty said, "here's are good and bad ways to use wikipedia" I think it had more impact.
But when faculty said, "here's are good and bad ways to use wikipedia" I think it had more impact.
November 24, 2025 at 3:30 PM
I think this strategy is also more likely to succeed. I remember when faculty used to say "no wikipedia." No one listened to them.
But when faculty said, "here's are good and bad ways to use wikipedia" I think it had more impact.
But when faculty said, "here's are good and bad ways to use wikipedia" I think it had more impact.
Definitely not one that the big players are likely to fix, no. And also, not something we as faculty can do much about, unfortunately. I think the best way forward is to provide students with LLM that *do* have those guard rails and strongly encourage them to use them.
November 24, 2025 at 3:30 PM
Definitely not one that the big players are likely to fix, no. And also, not something we as faculty can do much about, unfortunately. I think the best way forward is to provide students with LLM that *do* have those guard rails and strongly encourage them to use them.
That same task could be accomplished with google and stackoverflow, but would easily take 10 times as long.
November 24, 2025 at 3:28 PM
That same task could be accomplished with google and stackoverflow, but would easily take 10 times as long.
Most recently, for example, I forgot how to add text on top of a plot in ggplot for R. There are many ways to do it, and some are better and worse for particular use cases. I gave the LLM my use case and it said "here's the best way to do it."
November 24, 2025 at 3:28 PM
Most recently, for example, I forgot how to add text on top of a plot in ggplot for R. There are many ways to do it, and some are better and worse for particular use cases. I gave the LLM my use case and it said "here's the best way to do it."
I'm not sure there is divergence. I'm doing fairly basic things, and so are my students. We're all occasional coders who forget basic syntax. We don't need to learn a language well or do anything hard. We just need to remember what's the syntax for x, y, or z.
November 24, 2025 at 3:26 PM
I'm not sure there is divergence. I'm doing fairly basic things, and so are my students. We're all occasional coders who forget basic syntax. We don't need to learn a language well or do anything hard. We just need to remember what's the syntax for x, y, or z.
Who said I was being "uncritical?"
November 24, 2025 at 3:15 PM
Who said I was being "uncritical?"
It's very hard to know in advance how a given technological change will affect inequality broadly. I suspect LLMs are no different in this respect
November 24, 2025 at 3:14 PM
It's very hard to know in advance how a given technological change will affect inequality broadly. I suspect LLMs are no different in this respect
Yes. And this is evidence why you shouldn't just tell students "use LLMs to write your essay" (which is what the study did). That's an unproductive way to use it. And, honesty, I didn't need a study to tell me that
November 24, 2025 at 3:13 PM
Yes. And this is evidence why you shouldn't just tell students "use LLMs to write your essay" (which is what the study did). That's an unproductive way to use it. And, honesty, I didn't need a study to tell me that
One of the major findings for writing instruction is that they absolute best thing to do is write, get feedback, rewrite, get feedback, rewrite again, etc.
Whether LLMs can provide helpful guidance in the "get feedback" stage remains an open question, but there are some positive signs (I'm told).
Whether LLMs can provide helpful guidance in the "get feedback" stage remains an open question, but there are some positive signs (I'm told).
November 24, 2025 at 3:01 PM
One of the major findings for writing instruction is that they absolute best thing to do is write, get feedback, rewrite, get feedback, rewrite again, etc.
Whether LLMs can provide helpful guidance in the "get feedback" stage remains an open question, but there are some positive signs (I'm told).
Whether LLMs can provide helpful guidance in the "get feedback" stage remains an open question, but there are some positive signs (I'm told).
I almost never accept the rewritten version as is. BUT, it does get me out of whatever rut was keeping me from coming up with better phrasing. My students have found it useful in this way as well.
November 24, 2025 at 2:53 PM
I almost never accept the rewritten version as is. BUT, it does get me out of whatever rut was keeping me from coming up with better phrasing. My students have found it useful in this way as well.
For writing tasks, I find it more hit and miss. For professional writing, I use it exclusively for brainstorming how to rewrite sentences/paragraphs that I don't like. I'll feed it the paragraph and what I don't like about it, and I ask it to rewrite it (often I ask for more than one option).
November 24, 2025 at 2:53 PM
For writing tasks, I find it more hit and miss. For professional writing, I use it exclusively for brainstorming how to rewrite sentences/paragraphs that I don't like. I'll feed it the paragraph and what I don't like about it, and I ask it to rewrite it (often I ask for more than one option).
I should have added my own experience. Right now, I mostly encourage my students to use it for coding and help with very specific writing tasks. (That's mostly how I used it.)
It's very powerful for coding, and it's getting almost indispensable for me. My students agree.
It's very powerful for coding, and it's getting almost indispensable for me. My students agree.
November 24, 2025 at 2:53 PM
I should have added my own experience. Right now, I mostly encourage my students to use it for coding and help with very specific writing tasks. (That's mostly how I used it.)
It's very powerful for coding, and it's getting almost indispensable for me. My students agree.
It's very powerful for coding, and it's getting almost indispensable for me. My students agree.
You could say exactly the same thing but replace LLM with "professor." Shall we give up on professors too?
AI tutors long predate LLMs (especially in things like math) and one of the promises of this technology is to provide educational access in places without proper instruction.
AI tutors long predate LLMs (especially in things like math) and one of the promises of this technology is to provide educational access in places without proper instruction.
November 24, 2025 at 2:47 PM
You could say exactly the same thing but replace LLM with "professor." Shall we give up on professors too?
AI tutors long predate LLMs (especially in things like math) and one of the promises of this technology is to provide educational access in places without proper instruction.
AI tutors long predate LLMs (especially in things like math) and one of the promises of this technology is to provide educational access in places without proper instruction.
Those kinds of things don't have to be 100% accurate to be useful. Of course, they might be bad at this too, of course, but the reports of people who are using them has been mostly positive.
As I say, however, the proof one way or the other will be empirical
As I say, however, the proof one way or the other will be empirical
November 24, 2025 at 2:45 PM
Those kinds of things don't have to be 100% accurate to be useful. Of course, they might be bad at this too, of course, but the reports of people who are using them has been mostly positive.
As I say, however, the proof one way or the other will be empirical
As I say, however, the proof one way or the other will be empirical
The idea of using them as tutors is very different. They are asking questions ("have you considered...") or providing defeasible suggestions ("are you sure that you've talked about...").
November 24, 2025 at 2:45 PM
The idea of using them as tutors is very different. They are asking questions ("have you considered...") or providing defeasible suggestions ("are you sure that you've talked about...").
Whether they are effective at tutoring or not is an empirical question that requires empirical test to answer. The objections your raising are still focused on the mindset of them like search engines: providing answers which are taken as definitive
November 24, 2025 at 2:45 PM
Whether they are effective at tutoring or not is an empirical question that requires empirical test to answer. The objections your raising are still focused on the mindset of them like search engines: providing answers which are taken as definitive
I have a colleague who suggests that his students use it to explore ideas, especially for novel applications of existing mathematics. I am honestly surprised that he's having success in this domain, but he says it's not bad.
November 24, 2025 at 2:37 PM
I have a colleague who suggests that his students use it to explore ideas, especially for novel applications of existing mathematics. I am honestly surprised that he's having success in this domain, but he says it's not bad.
Will any of these ultimately work? I don't know. And neither does anyone else, honestly. That is my broader point: anyone who says "LLMs can't possibly have any good educational use" is overconfident at best.
November 24, 2025 at 2:35 PM
Will any of these ultimately work? I don't know. And neither does anyone else, honestly. That is my broader point: anyone who says "LLMs can't possibly have any good educational use" is overconfident at best.
There is some possibility of using LLMs for assessments. (Importantly, assessment is not necessarily the same as grading.) Especially for trying to do large scale assessments that would otherwise be prohibitively costly without them.
November 24, 2025 at 2:34 PM
There is some possibility of using LLMs for assessments. (Importantly, assessment is not necessarily the same as grading.) Especially for trying to do large scale assessments that would otherwise be prohibitively costly without them.
These projects are still in development, and may ultimately fail. But if they work, I think they could be a huge gain for students. Automated tutoring is something that long predates LLMs, and there is strong evidence they are valuable.
November 24, 2025 at 2:32 PM
These projects are still in development, and may ultimately fail. But if they work, I think they could be a huge gain for students. Automated tutoring is something that long predates LLMs, and there is strong evidence they are valuable.
I haven't used them myself, but I know people who are having success with bespoke LLMS that are trained to work more like good tutors: asking questions, making suggestions, but not doing the work for the students.
November 24, 2025 at 2:32 PM
I haven't used them myself, but I know people who are having success with bespoke LLMS that are trained to work more like good tutors: asking questions, making suggestions, but not doing the work for the students.
They also give bad ones, which I actually think is good as well... students have to think critically about the feedback.
November 24, 2025 at 2:32 PM
They also give bad ones, which I actually think is good as well... students have to think critically about the feedback.
Right now, I think they are moderately effective tools for helping with writing. They are imperfect, but if you upload a piece of a writing and a rubric they will give helpful suggestions.
November 24, 2025 at 2:32 PM
Right now, I think they are moderately effective tools for helping with writing. They are imperfect, but if you upload a piece of a writing and a rubric they will give helpful suggestions.