How I Built an AI Photo Search App With Cursor and CLIP
My first AI build searched local photo folders by description. The Cursor, Flask, and CLIP setup, the two rendering bugs, and what I changed next.
An AI photo search app can find an image from its contents, even when the filename tells you nothing. I built my first version with Cursor, Flask, and OpenAI's CLIP model, then searched a local photo folder with descriptions such as "trees" and "ice cream."
Do you have thousands of photos on your computer but no time to organize or tag them?
Do you find it nearly impossible to locate the one unique photo you wanted?
I don’t even take photos that often, yet I’ve accumulated thousands of them. Every now and then, I transfer them to my computer to free up space on my phone.
But once they’re there, they just sit in some mega-folder, staring back at me, while I feel completely overwhelmed and unmotivated to tidy them up.
Sometimes, I just want to find that one photo of a big lobster to share with friends or a picture of a gigantic ice cream to convince someone to visit that cute dairy shop.
But the problem is, I can never find them — they’re buried somewhere in the ocean of my photos.
To solve this immediate problem, I decided to start my first project: a content-based image search app. Thanks to generative AI tools and models, building this app feels more accessible than ever before.
Let’s dive in!
Setting the Scene: Installing and Setting Up Cursor
Cursor is going to be my main coding assistant throughout this challenge, helping me stay on track with projects and deliver code as fast as it could ever be.
My first step was to get Cursor up and running alongside VS Code, my go-to code editor. After downloading Cursor, I spent some time setting it up to work smoothly with VS Code, tweaking environment settings, and installing all the necessary packages.
Project Goals
The goal for this project was pretty straightforward. I wanted to build an app that could:
Display all images from a selected folder.
Let me search for images based on their content, not just filenames.
After some back-and-forth discussion with Cursor, I learned about CLIP, an amazing AI model developed by OpenAI. CLIP can match images with text, making it ideal for finding images that match a search query. Perfect!
Building the Basics: Initial Setup
Cursor really came to the rescue by generating code and organizing a neat folder structure.
In no time, it had:
Created a
requirements.txtfile with all necessary dependencies.Set up the main backend application file.
Built an HTML template along with JavaScript and CSS for the frontend.
Screenshot by Jenny
All I had to do was click through the steps, accept, apply, and run them as they popped up. Before I knew it, I had a basic app structure ready without manually coding everything from scratch. Cursor handled the heavy lifting—files, folders, dependencies, you name it.
It even listed all the steps I needed to follow to get the app up and running!
Screenshot by Jenny
A Few Hiccups
Of course, it wasn’t all smooth sailing. Here are a few areas where things got tricky:
Getting Familiar with Cursor
At first, I found myself clicking “Run” and expecting the program to execute, only to realize it was waiting for another step. Dependency packages even ended up in an upper-level folder by mistake.
It’s not a major problem though — just something you learn when getting familiar with a new tool. Knowing when to click twice versus single-click can be a small, but important, detail.
Image Rendering Issues
Initially, images weren’t rendering properly. I could see itemized objects in the UI, but they just wouldn’t display.
Image not displaying — screenshot by Jenny
Turns out it was an issue with path.relative_to(static_path), which wasn’t being generated correctly.
Once I fixed the path, I tested the "Search" button, but hit another snag: the search results didn’t display images the way the "Show All" button did.
After some troubleshooting, I discovered the problem was with Flask (the web framework) and its Jinja2 template engine.
While the “Show All” button rendered images server-side using Jinja2, the “Search” button needed dynamic updates without reloading the entire page. The solution? An AJAX request to update just the gallery section — saving both resources and time.
Final Outcome
After fixing the issues, my app was up and running locally at
Initial rendering — Screenshot by Jenny
The initial render works! I’d call it a success! 🎉
Testing the search function, I typed in “trees” — and it returned relevant images featuring trees.
Next, I tried “ice cream,” and there it was, a massive ice cream image right in the first spot! 🍦
I’m completely pleased with this first app that helps me search images by content. It required minimal coding effort on my part, yet it does exactly what I envisioned!
Questions, Ideas, and Future Improvements
As the app started working, I jotted down a few ideas and thoughts for improvements:
Flexible Image Folder:
Right now, all images are stored in astatic/uploadsfolder, but it would be awesome to allow users to select any folder on their device and have the app search within that. This would add flexibility and make the app a lot more versatile.Scaling the UI:
The UI code wasn’t component-based, which isn’t a big deal for a small project like this. But if I ever want to scale this app, a more modular, component-based approach would be essential.UI Improvements:
Speaking of the UI, it could definitely look better. I might bring in a library like TailwindCSS or Bootstrap for a quick facelift, but if you have any design tips, I’m all ears! ✨Low Matching Scores:
I noticed that match scores weren’t very high, even when searching for specific items that were clearly in the images. This left me scratching my head — was it a model issue or just the nature of image matching? (I later found out it indeed is the nature of image matching, but correct me if you know more!)
Final Thoughts
This first trial in my 30-Project GenAI journey has been a fantastic experience! From troubleshooting setup issues to building a functional content-based image search app, I’ve already gained so much insight into using AI for practical purposes.
Working with Cursor was also a huge help — having it assist with the code generation and structure really sped up the process.
Before this project, I had never used Flask framework, but now that I’m getting familiar with it, I’d love to try it more in future projects.
I hope you enjoyed this walk-through of my first GenAI project. Next I’d be sharing how I’m improving the app and stages of release!
If you have questions or ideas about this project, feel free to reach out. And if you’re considering a similar challenge, I highly recommend it — you’ll learn tons along the way!
A Little Background
What changed after the first version
The first version still required me to copy images into static/uploads. It also processed a small test folder, while the photo collection I wanted to search was much larger.
I took the next version in three directions:
Let the app open a folder I selected on my computer.
Process large collections in batches and show results progressively.
Separate the local image-search work from the web layer needed for distribution.
The photo-search optimization build covers the first two changes. The Image Finder launch story follows the same prototype into packaging, hosting, authentication, and payments.
The app eventually became a product, then I retired it. The local search remained useful. The distribution layer created a different class of work.