High-schooler Builds Test Program to Understand Genomic AI Papers
Hello everyone, Last year I wrote several posts on the large language models (LLM) of DNA sequences (evo/evo2 papers), and some readers emailed me asking about the plans to continue the series. Here is an exciting update for you. Guhan Thiagarajan, a high-school student, worked with me and built a “toy model” to help you explore these concepts.
Earlier Posts on LLMs of DNA Sequences
For context, these are the relevant blog posts from last year -
Discussing the Evo and Evo2 Papers
Evo and Evo2 - Math and Algorithm
Biological Aspects of Evo and Evo2 - Semantic Mining
Massively Parameterized Statistics
Training Approach in Evo and Evo2
More recently Google Deepmind also released its own genomic LLM on human variants, and I wrote about it in the following post -
Google Deepmind Solves All Genetics Problems
Building a “Toy Model” to Understand How They Work
For a simple description, you can think about the programs/AI models like evo and evo2 as chatGPT. You give a prompt, and based on it the program generates text. The only difference with chatGPT is that evo and evo2 generate DNA sequences and not English sentences. Of course, the prompts themselves are also DNA sequences.
But what do these generated sequences mean in the biological context? After playing with sequences generated by evo, I thought it would be wonderful to have what the physicists call a “toy model”.
At that time (May 2025), I met Guhan Thiagarajan, a really smart high-schooler, who loves math and coding. Guhan painstakingly went through all mathematical derivations of the evo papers and reimplemented the code in this library freely available from github. You can train it with sequences even on a Mac laptop.
There is more. Guhan and I decided to use the sequences from the plant plastids for training. Why plastids? It is because the plastids form a type of closed system. All plastids came from a single origin of a cyanobacterium being engulfed in a single-celled protist. Also, plastids genomes are tiny (~150K nucleotides), and therefore NCBI has plenty of genomes spanning a large evolutionary distance.
Another advantage of using tiny genomes is that the training cost is small. If you read the evo2 paper, you will find that the Stanford group authoring it worked together with Nvidia to train its large-language model on thousands of huge eukaryotic genomes. That process is expensive and does not leave room for experimentation with different subsets of genomes.
Guhan and I have been using his code to build DNA large-language models on various subsets of plastids, and then giving them prompts to understand the generated sequences. We have a number of interesting observations, and I will summarize them in the next post.
Edit. Here is the follow-up post - Sharing the Details on High-schooler’s Genome Large-language Modeler.
