Roger Kutyna
ML and applied AI engineer. M.Sc. Information Technology (Generative AI), Clark University. Bird photographer.
About
I'm an ML and applied AI engineer with an M.Sc. in Information Technology (Generative AI) from Clark University. I build practical systems that put machine learning to work, and when I'm not doing that you'll usually find me outside with a camera pointed at a bird.
Projects
Autolist AI - Agentic Multimodal Product Listing Generator
Autolist AI is an agentic tool for generating everything needed for an online product listing from a single image. This project was built for a class I took last semester called Generative AI Model Development. The project is built using n8n to create an agentic workflow and is hosted on a mini pc running linux that I use for headless hosting. The project begins with a front end with a form to enter a title, description, branding direction, and photo. Upon submission, this triggers a webhook and begins the workflow. The first step is to pass the image to Google’s Nano Banana (Gemini 2.5 Flash) which removes the background of the image, leaving only the product. This image of the product is then passed to GPT 5.2, which does a web search to find information on the product to ensure that the product listing is grounded in existing information on the internet rather than hallucination. Fortunately, this does not prevent listing of products that do not exist on the internet elsewhere and the model will do its best to provide what information it can. Based on the results of the web search, image, and the stated branding direction, the next node uses GPT 5.2 to create a comprehensive branding direction and style that ensures the images and video generated are consistent. With this background done, the next steps begin. These steps are the image and video choreographers. GPT 5.2 is used for both and takes in all previous info and creates three prompts, each passed via structured output, to Nano Banana which generates an image based on the original product photo, background info, and photo prompt. The video creation works much the same way, a video prompt is created and passed to Sora 2 which generates the video. In addition to this step, GPT 5 Mini is used to create a product description based on the previous information. Finally, the images, video, and description are sent via the response to the webhook node to the front end where the description can be copied and the images and videos can be downloaded. I would love to host this publicly to let anyone try it, however it gets quite expensive due to the video generation that costs $0.10/second. If I were to implement this project in a production environment rather than for a class, I would make a number of changes. The primary flaw with this product is its fragility. If an API returns an error which happens often, the workflow will simply stop and need to be restarted. In order to fix this issue, I would instead break each piece into their own workflows that will auto retry if an API error is received. Each of these would then save to disk to ensure that the results remain. This, combined with a move to asynchronous execution that would allow the user to queue jobs and be notified when they complete, would allow each job to become much more robust and only execute each step once. In addition, many of these steps, such as the writing of the description, can be done with local models that would reduce API cost and likelihood of errors. I would also ensure that the use of this product is secured through a log in and only available within company networks. This, combined with rate limiting of API calls would reduce the risk of a surprise API usage bill. In this same vein, I would add a dashboard to be able to centrally visualize usage and handle errors. One last change that I would make is the ability to regenerate specific parts of the output without running the whole workflow again, or even giving an AI Agent the ability to look over the output for quality and regenerate pieces if necessary. This would ensure quality and would allow the user to replace one photo or the description without running the whole system again and costing API usage.
BirdBrained: Automatic Bird Species Identification and Photo Tagging
Intro BirdBrained is the project that I have come up with for my capstone project for my Master’s degree. This is a project that is motivated by my need for photo library organization. Since I have gotten into bird photography, I have taken well over 30,000 photos (just since August of ‘25!). I was previously using iCloud photos which includes keyword search which automatic classification of photos, but since switching to local storage and Adobe Lightroom, I have been missing this feature. Pipeline: My knowledge of machine learning and AI gave me the idea to train my own bird species identification model that I can run locally to serve this function. This project is still in progress and I still want to experiment with several classifier models for a final write up with my final learnings. That being said, this project has been a huge success so far. The project is built on a pipeline that begins with YOLOv11 from Ultralytics that performs subject detection in each image. The subject detection model also creates bounding boxes for the subjects labeled “Bird”. The original image is cropped to this box and this cropped image of the bird is passed to the species identification model. The species identification model, based on ResNet-50, classifies the species. The cropped portion containing the bird is then passed to a sharpness scoring method that is based on using a Sobel Operator to calculate the Tenengrad energy of pixels which measures changes in brightness to do edge detection. Instead of measuring this score across the whole cropped section, it works in patches that are 10% of the image and move across the image. The top 5 patches according to Tenengrad energy are then averaged to produce a final sharpness score which is normalized to the sharpness scores of all photos from an inference run. Finally, the images are tagged with the predicted species and a confidence and sharpness score (low, medium, high). These are searchable keywords in Adobe Lightroom and enable easy library management. Training: When it comes to the choice of models for this project, I wanted something that was lightweight enough to train on my laptop (Macbook Pro with an Apple M3 Pro and 18 Gb of memory). With this constraint in mind, I chose a ResNet-50 that was pretrained on ImageNet. This model is relatively lightweight and has the potential to achieve high quality results. ResNet-50 is a 50 layer deep convolutional neural network developed in 2015 by Microsoft. A convolutional neural network is a neural net with a small filter that moves across the input to produce a feature map across the image to be passed through the rest of the network. A revolution that ResNet ushered in is the concept of residual connections that help to alleviate the vanishing gradient problem - changes from backpropagation get smaller and smaller the more layers it goes back - by ensuring that the gradient continues back when it would otherwise be reduced to near zero prior to reaching the early layers. This is a concept that has continued in machine learning and has been adopted by the transformer models that have revolutionized the field. The dataset that I am using for this project is a dataset of 48,000 labeled bird photos that consists of 404 species that are found in North America. This dataset was released in 2014 and is coordinated by the Cornell Lab of Ornithology for computer vision use: https://dl.allaboutbirds.org/nabirds. For my initial testing, I used a subset of 98 species of birds that are found in New England and would cover the majority of birds that I have photographed while enabling much faster training times. The first step of Fine-Tuning on this dataset is to unfreeze only the Head (final layer) which allows the pattern recognition that has been trained on ImageNet to be adapted to the bird species dataset. Next, Layer4 is also unfrozen at a lower learning rate to ensure that the beneficial high level feature recognition is not overwritten. Layer4 is able to learn the high level features necessary for the task of bird species identification. Lastly, Layer3 is unfrozen with a lower learning rate to allow for mid-level features to be adapted to the species identification task. Initially, I was able to achieve an accuracy of around 80% on the dataset using the supplied train/test split, which was 50/50. I was able to improve the accuracy to around 85% by simply changing the train/test split to 80/20 while ensuring that this ratio was maintained across the species within the split (80/20 per species). At this point, the primary issue that I was running into was overfitting. While 48,000 images may sound like a large number, this is actually a very small dataset when it comes to modern-day machine learning models, especially with more than 400 classes. This meant that the model would optimize too much for the training split, achieving an accuracy of over 99% while struggling to approach 90% on the test split. To combat this, I first experimented by modifying the learning rate and batch size, however these didn’t have much impact on this issue. I also tested a range of weight decay values which adds a loss penalty based on the magnitude of the model weights, but this didn’t appear to help. The most effective modification in initial training was label smoothing, which lowers the value of the correct class value by some value to again help prevent overfitting. A value of 0.05 allowed the model to consistently hit an accuracy of ~90%. After the revelation of a label smoothing, I went back to experimenting with the learning rate by lowering it, again to prevent overfitting. This was reasonably effective, bringing the accuracy to 91.6%. The most recent improvement has been through the implementation of a suite of dataset augmentation features to help combat the small size of the dataset. The first of these is RandomResizedCrop, which randomly crops a sub region of the image area and resizes it back to the image size of 240 by 240 and helps to ensure the model can recognize birds at different scales and position while introducing greater variation into the database. The second feature is RandomHorizonFlip which horizontally mirrors the image 50% of the time, again increasing the effective size of the training dataset. Third is ColorJitter, which modifies brightness, contrast, saturation, and hue by small amounts to vary the images to match potential variation in real photos and different shooting conditions. Fourth is RandomErasing, which creates a random black rectangle that covers 20% of the image for 30% of images to help generalize further and better handle situations where a part of the bird is obscured. The last modification is an addition of Dropout which randomly zeros some portion (in this case, 40%) of activates between Layer4 and the final layer. This forces the model to learn to be more robust by identifying species based on a variety of features rather than a smaller subset. These further modifications have allowed me to reach an accuracy of 93.11% on the base dataset, 93.2% Top-1 and 100% Top-5 accuracy on my personal test dataset. Next Steps Next, I intend to implement MixUp or CutMix to further help reduce memorization of the training dataset. I also would like to implement learning rate scheduling and further experiment with learning rates and the number of epochs of each training stage now that the issue of overfitting has become less of a roadblock. I would also like to create a larger personal labeled test dataset to be able to get a more accurate picture of the performance of this model. Up until this point, this has been done on the subset of 98 species, so I would like to attempt to train the model at the full 555 species to see how the accuracy compares. Lastly, I would like to do similar experimentation on other models, including at least one vision transformer, to compare performance. I do not have much budget for compute, but I would like to rent compute time on a remote server to make the final model as accurate as possible. I would also like to attempt to fine-tune the YOLO model for the purpose of being more accurate at bird identification, however it seems to be quite accurate and this is not the primary bottleneck for the project. It would also be beneficial to experiment more with the sharpness model and attempt to create a ML based model to score sharpness or specifically score sharpness on a bird’s eye, but again, this will come after further experimentation with the species identification model. To further explore the implementation of this project or to attempt to run it, check it out on my GitHub: https://github.com/rkutyna/BirdBrained
Blog Posts
Blog #3: Automating Bird Species Identification Model Experimentation
This update to my BirdBrained project is one that I am particularly excited about. It is an automated agentic experimentation loop that is inspired by Andrej Karpathy’s autoresearch project (https://github.com/karpathy/autoresearch). Karpathy’s autoresearch project is based on the idea that an agent is capable of running experiments to improve an AI model if given the capability to do so. Within his project, a program.md file is used to instruct the agent on how the setup works, prepare.py is the file that prepares the environment and dataset, and train.py is the file that facilitates the experimentation and gives the agent access to all parameters and potential modifications. Train.py is the file that the agent actually modifies to change the experiment on each iteration. In Karpathy’s implementation, this is used to train a small LLM on an Nvidia H100. Setup: I decided that this concept would be worth experimenting with on my own model training project: the bird species identification model that is part of my BirdBrained project (https://rogerkutyna.com/projects/8). In order to do this, I used Claude Code to create a framework similar to Karpathy’s project. It consists of four files: program.md, autorun.py, prepare.py, and train.py. Prepare.py handles preparing the NABirds dataset as well as train/test splits. Autorun.py contains the iterative framework that facilitates each experiment. It starts by saving the training configuration in train.py, building the prompt for the agent from program.md, launching a codex exec instance, and keeping or restoring the train.py file. The train.py file contains the actual logic for fine-tuning the model and all hyperparameters and augmentations for the agent to tune before a training run. The modification of these parameters is done by the agent, which in this case is GPT-5.4 that is called through a codex exec instance. This agent is given the contents of program.md as a prompt, the training history, and is instructed to test one hypothesis to improve the accuracy of the species identification model. It then runs the train.py file and monitors the training performance. When the run is completed, if the accuracy increases, the updates the agent made to the train.py file are incorporated into the file as the new default, the agent writes short notes on the training performance, and the results are saved in a csv file with previous runs. Each run is limited by a time limit to ensure that they are comparable and take a finite amount of time. The loop runs for a number of iterations set with a CLI flag. Results: I ran this training loop both for the 98 species subset that I used for initial training and the full 555 species dataset for a total of 71 runs over the course of several days. On the 98 species subset, the agent-run training was able to achieve a top-1 accuracy of 96.31% on the test split, 94.6% top-1 accuracy on my personal test set, and a 100% top-5 accuracy on my personal test set. This is an improvement of 3.20% over the result I was able to achieve training the model myself. On the full 555 species dataset, the agent-run training was able to achieve a top-1 accuracy of 91.91% on the test split, 88.3% top-1 accuracy on my personal test set, and 100% top-5 accuracy on my personal test set. This is an improvement of 8.26% from what I was able to do in the small number of training runs that I did with the full dataset. What is most interesting to analyze are the changes that the agent made. On the 98 subset runs, it was able to reach ~93% accuracy by just modifying the learning rate and adjusting the number of epochs for each stage of training. Implementing test time augmentation (a simple horizontal flip of the image) brought this accuracy to 95%. The next improvement was through the implementation of GeM (Generalized Mean) Pooling. The base ResNet-50 model contains a global average pooling layer after the last convolutional layer that takes the average of a 7x7 grid for each of the 2048 channels. Each channel represents a feature, so the average of this grid is basically how present is the feature learned by this channel. GeM Pooling changes this from the average to GeM(x) = ( mean of x^p )^(1/p) where p is a learned value. A higher p value will mean that just the peak activation will survive, while a p of 1 weighs each location in the 7x7 grid equally. This allows the model to learn whether it should pay attention to the most highly activated regions of a channel (feature) or if they should be considered the same. This helps with bird species identification as the difference between species is often in specific areas, so weighting the highest activation can help to distinguish between otherwise similar images. This feature further brought the accuracy up another 0.2% to 95.19%. Training was initially capped at 20 minutes, then 30 minutes, and the final runs were capped at 40 minutes. The increase of time along with the addition of the ability to train all layers led to the final accuracy of 96.31%. On the full 555 training runs, the time was set at 60 minutes and was later increased to 120 minutes to handle the much larger dataset. This training also began with the best training configuration from the 98 species subset, so it had less modifications to make that weren’t already implemented. In the first several runs, it was able to achieve an accuracy in the mid 80% range, which was on par with my previous runs. The primary gains came from minimizing the time spent training the head (final layer) and layer 4. Going straight to layer 4 yielded the best results thus far, bringing the accuracy to 88.9%. The agent continued to tune the stages and learning rate and was able to reach an accuracy of 90.1% before extending the time limit to 120 minutes. This increase, along with the addition of trivial augment (randomly selects one operation from a broad pool including color adjustments, sharpness, rotation, and translation, applied at a random magnitude) brought the accuracy up to its final value of 91.91%. This is a great result, but I have yet to experiment with the unfrozen layer 1 and 2 training using the full 555 species dataset, so I will do a couple runs with a 240 minute time limit overnight and update this when I have those results. Conclusion: This concept of autoresearch has been tremendously successful. It was relatively easy to implement and having GPT 5.4 run experiments has produced good results. I think that I would have been able to reach the same accuracy given the same number of training runs, but the ability to set it and forget about it while it experiments is very convenient. It also allows it to pivot in a way that queueing a large number of runs and letting them run before checking results would not. I am certainly interested in exploring this concept in the future with different types of models and higher end hardware, but for now I am very pleased. Next Steps: Given that this has produced excellent results thus far, I am excited to keep experimenting with it. The primary thing that I would like to do next is to explore using different models, such as a vision transformer, for this same task using the autoresearch concept. I will also plan to purchase an Nvidia RTX 3060 12gb that will be a large step up over my Apple M3 Pro for training runs like this. Either that, or I will rent compute time on a server for the same purpose.
03/17/2026, 8:03 PM
Blog #2: Claude Code Diagnosing System Failures
This morning, I went to start applying to jobs and realized that my OpenClaw agent hadn't sent my daily briefing. I checked this website too and it was down. Definitely a nerve-racking moment - I'm applying for jobs and my portfolio website is down! I tried to access the machine that these services are running on via SSH and was unable to reach it. This machine has been a little finicky before, so I wasn't totally surprised, just annoyed. Fortunately, I was able to get it back up and running by manually restarting the computer. However, this didn't solve the root cause obviously. I've seen a lot of hype recently about Claude Code and Codex and had good experiences with both building apps and small projects recently, so I decided to let Claude try to fix it with this prompt: "This is my mini-pc that I have running a web server, openclaw, and n8n. It was working yesterday, but last night around 7-8 pm it stopped responding. I just noticed this morning and had my partner restart it via the power button, and now it's back up (I access it via ssh and tailscale, so I was locked out until it restarted). Given that this hosts my portfolio and i'm applying to jobs, I need it to be up. Can you investigate why this happened - check whatever logs necessary, etc. It's running ubuntu server 25.10. Please ask me any necessary questions to help or if you need me to help access something on the computer" Claude explored a variety of logs and eventually was able to find the root cause - the WiFi driver disconnecting and failing to reconnect. Beyond just identifying the problem, Claude created a WiFI watchdog program to check that the gateway is up every 30 seconds and restart the driver if it is down for more than a minute. This should resolve the problem going forward. I have also set up an account with uptimerobot.com to monitor my website and notify me if it goes down in the future. You can check the public status website here: https://stats.uptimerobot.com/S49FWeGbP1 This experience is just one of many recently that have made me very optimistic for the progress of LLM based Agents. I know that there are many complicating factors for doing this same work in a production setting - large code base, security concerns - but I’m still impressed.
03/12/2026, 1:35 PM
Blog #1: OpenClaw
After seeing quite a lot of coverage on OpenClaw - much of it about the security concerns and the hilarity of Moltbook, I finally decided to try it. In order to mitigate some of the security concerns, I opted to install OpenClaw using Docker on my mini-pc that I use for hosting a website along with a couple other small services. After a little bit of trial and error, I was able to get OpenClaw running with interaction via Telegram and linked to Anthropic to use Claude Sonnet 4.6 via the API. So far I have simply asked it to give me a daily briefing on daily AI news and one AI/ML research paper at 8am daily. This is certainly useful, but what has amazed me most is its ability to fix itself. I noticed that it was using a tremendous amount of tokens on my Anthropic API key (Expensive!). When I checked the console, I realized that it had failed at creating the briefing - an error with web search I believe. But it had simply tried a different method and successfully completed the briefing. Another cause of the excessive token usage is because it had created a briefing using the heartbeat and using a cron job, so it was creating two briefings each day and only one was reaching me. All I had to do to fix it was tell it to investigate the issue and fix it and it just did it! I've become used to tools like Codex or Claude Code being good at fixing issues in other programs, but this took it to another level and I look forward to a day when these tools are reliable and secure enough to fully manage my computer. I will certainly keep experimenting with this tool and I'm happy to hear suggestions on how to use these tools, just shoot me an email! [email protected]
03/12/2026, 1:34 PM
Contact
The fastest way to reach me is email. I'm always happy to talk about ML, engineering work, or birds.