My ADHD precludes me from giving flat rates, which would require predictive abilities I do not posses - I typically give a (very wide) interval for how long I expect a job to take, and always charge for the actual time dedicated to the job (typically some money upfront). It's worked well so far 🤷‍♂️

if I was a confounder I would simply be easy to observe

But yeah, z scores

This seems contradictory to the previous skeet? Why should I "take the hit" in X? Unless you're saying charge them both for two weeks of work?

(that was a lot, and ignoring some empirical issues -)

If we truly are headed into a post literate future, how can we still (as a society) support (higher) education system? The math ain't mathing - people don't have the skills or attention to learn, yet I am expected NOT to fail entire courses??

I'm not worried about "what will be left to do" - I'm worried about the "will we still be able to learn interesting and complicated things" when LLMs are so clearly devastating development of learning skills.

Students don't learn how to teach themselves - that skill is also outsourced to LLMs 😕

This is already happening, with companies offering AI+SEO services - how to build a site that is more appealing to LLMs, etc.

I see you share my enthusiasm

Down the line this means there will be a shortage of experts at various intermediate levels. Which sucks for science all around.

Am I being pessimists bcause all but 1 of the papers I graded today used 0 of the methods I taught in class, using instead generic versions of similar methods? ... 😔🤷‍♂️

3) This new future will lead to hard self selection of experts

Since we are swimming upstream trying to develop students' skills, only a handful of bright-eyes-bushy-tailed students who are actually interested will actually learn to do the thing.

Most students were never interested in skill devpmnt, but were coerced to as it was to only way to get a passing grade, & educators took advantage of that.

Now, truth is an LLM in the hands of a mediocre student w/some good soft skills can get a passing (low) grade in my courses 9 times out of 10.

Since I can't force students to deliberately engaging in practice (the kind that (1) is antithetical to "working with chatGPT", (2) leads to skill development), I might as well not focus on that. I can trap them in a room w/o a silicon brain & force them to come up w/ answers to critical questions.

2) We probably shouldn't bother teaching "how to do".
Instead we should double-down on understanding and thinking critically about these methods.

Gone are the days where we could teach ideas and trust students to properly practice the "how to"...

1) Back to pen and paper

I hate tests. As an educator, they don't make sense to me - they have low ecological validity: when people really engage with data, they're not trapped in a room without a computer and only 3 hours to get the job done.
But now there's no other choice, partially because -

Here are some thoughts about higher education - specifically statistics & research methods - after grading some more papers written with "assistance" from LLMs.

🧵

So I don't know about how you use LLMs - but the trend I'm seeing from where I am positioned is that the academic population is becoming less knowledgeable, less critical (!), and overall less engaged in their own work.

To me, that's as bad and boring as it is inevitable. 😔

I agree that LLMs are helping non native English speakers write more polished papers, write letters, ...

But if we keep "letting" it do more and more things for us, what are we actually doing anymore? Isn't brainstorming the part we're suppose to do?

Any metric is game-able :/

I don't have a good answer - but that doesn't excuse a bad one IMO.

I also don't know if I agree GPT is helping you.
I mostly interact with students - it definitely helps them be stupid faster, but (and!) it also hiders their advancement of out novice-level abilities.

Sure, but these aren't just "flawed metrics" - there has been extensive debate and empirical work over decades showing that these metrics are bad 🤷‍♂

Even if you're right (which you certainly are to at least some extent) - I would be hesitant to say this "improves academic performance", which was what my original response was saying.

I would definitely not recommend chatGPT for lit review. I do use elicit.

They look soooo good!