Full program details are available at: conf.researchr.org/home/icst-20...

Today we had a successful 8th IEEE Int’l Workshop on Next Level of Test Automation (#NEXTA2025). Highlights included:

✅ KEYNOTE by Serge Demeyer
✅ TALKS
- Influence of Pure & Unit-Like Tests on SBFL
- Witness Test Program Generation
- Adaptive Testing for LLM-Based Applications
✅ DISCUSSION!

Looking for evidence of data leakage in the #HumanEval code generation #LLM benchmark? Check out our analysis of a subset of HumanEval tasks vs comparable tasks on #ChatGPT, #Claude and #Llama. @riddhimore.bsky.social

Human Brain vs. LLM Agent
🧠 vs. 🤖

Hardware…
Consciousness…
Learning style…
Creativity…

- Bang Liu, #SANER2025 Keynote

The Large Language Model (#LLM) Lifecycle from #SANER2025 keynote by Dr. Bang Liu, University of Montreal

➡️ Pre-Training (e.g. data synthesis, model architecture)
➡️ Post-Training (e.g. instruction tuning, bias mitigation)
➡️ Serving (e.g. cloud deployment)
➡️ Utilization(e.g. prompt engineering, RAG)

Yesterday Riddhi More presented our paper “Assessing Data Augmentation-Induced Bias in Training and Testing of Machine Learning Models” at the 1st Int’l Workshop on Fairness in Software Systems (#FAIRNESS2025) in Montreal, Canada. 🇨🇦
Paper preprint, slides & code/data available at: bit.ly/4brEBJm

“Code generation is just the tip of the iceberg towards automated software engineering!!” - Dr. Baishakhi Ray (Columbia University), #SANER2025 Keynote

A great keynote by Foutse Khomh (@swatlab.bsky.social) on “Navigating Fairness Challenges in Intelligent Software Systems” at #FAIRNESS2025. I appreciated the emphasis on #AI model transparency, auditing and governance as essential to the development of fair systems that mitigate bias.

Thank you to the organizers of the #AI Meets Software Quality Int’l Colloquium for inviting Riddhi More & I to speak on Monday. If you're interested in #LLMs & #flakytests the preprints of the papers presented are available at bit.ly/40OvYUa (FlakyXbert) & bit.ly/4jILzNT (augmentation bias).

Our paper "Addressing #DataLeakage in #HumanEval Using Combinatorial Test Design" has been accepted for publication at the Int’l Conference on Software Testing, Verification & Validation (#ICST2025), Short Papers, Vision & Emerging Results.
co-author: Riddhi More
arxiv.org/abs/2412.01526
#benchmark

I'm happy to share our paper "Assessing Data Augmentation-Induced Bias in Training and Testing of Machine Learning Models," has been accepted to the 1st International Workshop on Fairness in Software Systems (FAIRNESS 2025), co-located with #SANER2025! #ontariotech @riddhimore.bsky.social

Are you working on #softwaretesting & passionate about test #automation and #AI testing? If so, consider submitting to the 8th IEEE Workshop on the Next level of Test Automation #NEXTA2025, co-located w/ #ICST2025
Submission Date: Jan 10, 2025 [UPDATED!]
bit.ly/nexta2025

I’m excited to share that our paper "An Analysis of #LLM Fine-Tuning & Few-Shot Learning for Flaky Test Detection & Classification" has been accepted for publication at the Int’l Conf on Software Testing, Verification & Validation (#ICST2025). This paper is first-authored by @riddhimore.bsky.social

I had a great time today attending the #OntarioTech Cakes & #AI Winter Show-And-Tell event. A great opportunity to engage with researchers from across the university and establish interdisciplinary collaborations. Thanks to @petelewis.bsky.social for organizing! 🍰

Thank you to Prof. Akramul Azim for giving a guest lecture today in my “Applications of #LLMs to Software Engineering” grad course. He talked about his #CASCON2024 paper “FaultLLAMA2: A Fine Tuned Large Language Model for Code Recommendations to Reduce Java Lock Contention” #OntarioTech #concurrency

It was great yesterday to have my colleague Cristiano Politowski speak to our graduate students and share his research experience at the intersection of Software Engineering and Game Development. #OntarioTech #softwareengineering #gamedev @crispolito87.bsky.social

Are you working on #softwaretesting & passionate about test #automation, test generation, and #AI testing? If so, consider submitting to the 8th IEEE Workshop on the Next level of Test Automation - #NEXTA2025, co-located with #ICST2025

Submission Date: Jan 3, 2025
More Info: bit.ly/nexta2025

I was happy to have Prof. Pooria Madani give a guest lecture today in my Applications of LLMs to Software Engineering graduate course. He presented “Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats” — arxiv.org/abs/2410.22293
#OntarioTech @pooriamadani.bsky.social

I’m happy to now be on 🦋! 👋🏻
I’m a CS professor at Ontario Tech University in Oshawa, Canada. 🇨🇦 At #OntarioTech, I lead the Software Engineering & Education Lab (www.seerlab.ca) where we research applications of #AI, #NLP, #LLMs to #softwareengineering, #softwaretesting & #CSEd.