Building A Local Clinical AI Note-Taking Tool As A Psychology Grad Student

Yes, I’m a clinical psychology graduate student with two young children, an ongoing dissertation, and I’m still waiting to hear about internship on Match Day. Naturally, I decided that was the perfect time to start building a software product and, somehow, a company, with minimal coding experience.

And yet, here I am.

Part of that is timing. AI has been genuinely transformative for me. It’s helped me lean into areas where I’m already strong and compensate in areas where I’m not. Everything in between, like the areas where I'm neither strong nor particularly weak, is another story, and that's probably where AI should stay out of the way.

In early summer 2024, OpenAI released a new model and my brain more or less exploded. It wasn’t just that it was better. It was the pace and acceleration. I realized pretty quickly that this wasn’t something I wanted to watch from the sidelines. I wanted to understand it, which meant using the tools, breaking them, and seeing where they fit into my life.

So I started following releases closely, trying new models as they came out, and getting familiar with what they could and couldn’t do. When vibe coding (AI-assisted coding, even for people who aren’t programmers) started taking off in 2025, I jumped in. At first it was small, disposable projects. One-off tools, like pdf editors or mini psych-oriented games for a class.

By the summer, I started working on my first substantial project: a PHI detector and report de-identification tool. I learned some hard lessons along the way and eventually pivoted away from that project. I was interested in seeing what I could do with AI-assisted coding, and even solve a real-world problem for myself along the way. I wanted a tool that fit into the way I already worked and put everything in one place.

My documentation workflow had been fairly stable for a while with brief shorthand notes, individual assessment recaps, and then turning those into polished notes and reports. I had already refined that process using different custom GPTs, small projects, and other one-off setups, but all of that lived across separate tools and workarounds. What I didn’t have was a single system that could handle the full workflow while also being HIPAA compliant and allowing PHI, especially for report and intake writing where that information plays a more central role.

I had seen cloud-based solutions, but for me, the problem was practical. I was already using AI in ways that avoided PHI by design, which is inherently a bit risky and, I suspect, somewhat common among clinicians. I also didn’t have much room for yet another subscription, especially when most of the tools I was using were discounted or free because of my student status. The idea of building a local note and report writing assistant started to make sense. It addressed privacy concerns for people who care about that, but more importantly, it solved my immediate need without introducing another service I had to manage.

So I began working on vibe coding what would become LocalScribe, still unsure whether I could actually pull together something that functioned as a real product. I started with the UI, mostly because that was the most concrete place to begin and gave me something visible to react to. That part was engaging, but it didn’t answer the bigger question of whether the rest would hold.

The more interesting part was testing local AI models. I had assumed, based on small language model benchmarking and general discourse at the time, that local models would be noticeably limited. Instead, I found that even relatively small models running locally were exceptionally good at the specific task I cared about, which was taking existing material and organizing it into coherent clinical notes and reports. That included longer narratives, structured sections, and correctly copying long lists of testing scores.

Progress was gradual, but it was deliberate. As the core note-writing workflow started to stabilize, I became more confident adding features that would make the tool genuinely useful, things like dictation, attachments, and even a report deidentification option. At the same time, the underlying AI landscape was changing quickly. Models were improving month by month, access to compute was expanding, and workflows that had felt clunky early on became easier to manage. That combination made it possible to keep moving forward, even when the end state still felt unclear.

As more of the pieces came together, the project started to feel usable rather than tentative. The core workflow worked reliably, and the surrounding pieces were stable enough that I could focus on refining them instead of constantly fixing them. At that point, I started having conversations with trusted friends and advisors, along with a lot of back-and-forth with AI chat tools, to think through what it would look like to offer something like this responsibly. Those conversations tended to center on similar themes of what safeguards actually mattered, what risks were real versus theoretical, and whether this was solving a problem that other clinicians would recognize as their own.

This is a different way of thinking about what’s possible. If someone has a real problem they want to solve, they can now build something to address it. That might be a quick prototype, a small tool that helps a few people in an office, or something designed for a very specific clinical use case. Larger projects are possible too, but they no longer require starting from scratch or fitting yourself into a predefined role. I’ve been able to work this way, and my hope is that this makes it feel more accessible to others. We don’t have to stay boxed into narrow roles. We can work across boundaries, collaborate more easily, and come up with solutions that actually fit the problems in front of us.