Like many iPhone users, I’ve spent years snapping photos—of family, friends, travel, food, and the occasional blurry pet mid-jump. My photo library had become a sprawling, unsearchable archive of life. That’s when I decided to build PhotoNester—a smart, on-device photo classification app that automatically organizes your photo library into clean, scrollable albums.
Here’s how I went from a rough idea to a working, App Store–ready product.
The Spark: Organizing Chaos
The idea came from a personal pain point. I had thousands of photos backed up from iCloud, but no good way to make sense of them. I’d tried Apple’s Memories feature, third-party gallery apps, and even manual sorting (briefly—never again). Nothing gave me the control or structure I wanted without compromising privacy.
That was the moment it clicked: What if I could use AI to group photos by content—travel, pets, nature, documents—right on the device, with no cloud processing involved?
Choosing the Brains: CLIP as My Visual Engine
I needed a robust model that could understand visual content well enough to group similar images together without hardcoded labels. I chose OpenAI’s CLIP model, which generates high-quality embeddings from images. These embeddings are like fingerprints—vectors that describe the semantic content of a photo.
To run CLIP on iOS, I converted the PyTorch model into Core ML format using coremltools, optimizing it for on-device performance. I then tested it across a variety of image sets—vacations, work screenshots, family albums—to ensure the embeddings were consistent and meaningful.
Core ML and Swift: The Technical Build
I built the app in Swift, using SwiftUI for the interface and Core ML to run the model. I wanted the experience to be clean, responsive, and native. One of the key challenges was batching and optimizing image processing. Running neural inference across thousands of photos on a phone can tank performance (and battery).
So, I implemented:
- Batch processing with memory safeguards (processing 100 images at a time)
- Progress tracking with visual feedback to the user
- Persistent state, so users can stop and resume without losing progress
The classification is unsupervised—images are grouped based on visual similarity. Then, using k-means clustering on the embeddings, I sort them into labeled albums like “Travel,” “Pets,” “Food,” etc., which users can rename or merge.
Keeping It Private
A big goal was to ensure privacy. PhotoNester never sends photos to a server or uses the cloud. Everything—classification, clustering, album generation—runs directly on the device. This was not just a security decision but a performance one too. Users see near-instant results, and their content stays theirs.
The App Store Hurdles
Submitting to the App Store was its own journey. Apple initially rejected the app for a crash on iPadOS 18.5. After digging through logs, I found that the MPSGraph backend was unavailable on that iPadOS version, and the model fallback wasn’t configured correctly. The fix? Explicitly set MLModelConfiguration().computeUnits = .all to allow CPU/GPU fallback.
Once that was resolved, I finalized the App Store metadata, created visuals for the feature list, and submitted again.
What PhotoNester Does Now
Today, PhotoNester automatically groups your entire photo library into intelligent albums—Travel, Pets, Food, Nature, Selfies, Documents, and more. It works offline, respects your privacy, and feels like magic when it turns a jumbled camera roll into organized memories in just minutes.
What’s Next
There’s more coming—custom category creation, time-based filtering, smart tagging, and better album suggestions. But for now, I’m proud of what PhotoNester has become: a fast, private, beautiful way to rediscover your photos.
If you’re tired of endless scrolling to find that one beach pic or that concert selfie, give PhotoNester a try. It’s the photo app I always wished someone else had built—so I did it myself.
