When the algorithm says yes

So, I had a moment this week. The kind that starts with a letter in the mail and ends with you staring at the ceiling, wondering what you just read. It's a doozy.

I received a letter on behalf of my mom about a healthcare program she'd been enrolled in, one I'd never heard of. I called Medicare. They had no idea what it was. I called her primary care physician's office, since their name was attached to it.

When I expressed concern about how her sensitive data was being handled, the person on the line told me an algorithm likely added her.

I asked, "So what you're telling me is there was no human in the loop?"

Cue the immediate backpedal: "Actually, I don't know the process. I don't know if an algorithm was involved." 😳

I've written before about what happens when the algorithm says no. I didn't expect to be writing about what happens when it says yes, quietly, on my mom's behalf, with no one able to explain how.

It felt violating. Like someone decided they knew what was best for her, and never thought to ask.

When a lot of folks hear terms like "technology," "algorithms," or "AI," it's so easy to just shut down and feel helpless. The concepts feel nebulous, cold, and entirely out of our control.

And real talk: That reaction makes total sense.

The incident sparked a totally half-baked, open-ended thought for me: What if we don't just view technology as this inevitable thing happening to us, but as something we can actively question and shape?

Listen. I'm not out here advocating for throwing more tech at our problems. But here's what's been simmering in my brain.

Alongside big-picture regulation, maybe the key is building up our own understanding of the process. If we know the basic mechanisms, we, as folks and caregivers, can better advocate for ourselves. We can know exactly who to contact, what questions to ask, and where our data is going and for what purposes.

Maybe the better question isn't whether to trust the tech, but whether we can shape it. Instead of a tool that distances, frustrates, and confuses us, could ethical AI and human-in-the-loop systems actually be protective? A world where technology helps catch the cracks, but human agency always holds the safety net?

And while we're here: can we talk about the rise of AI-focused degrees for a second? I admit I'm sometimes guilty of rolling my eyes when I hear about them (I'm assuming a lot and we know what that does 🙃), but now I'm wondering if they might actually be part of the solution. As youth choose their own adventure (if that brought back memories of the books, hi, friend! ☺️), they are in a unique position to lead and reimagine how tech works to protect communities.

I wonder: do these degrees include a deeply human, ethical, protective component? Or is it all still focused on scale? Having worked for a tech startup myself, scale always seems to be the ultimate goal. What would it look like if, instead of the well-known tech mantra to "move fast and break things," a new ideal took over: "move thoughtfully and improve things"?

To my fellow caregivers, advocates, and the folks with curious minds: Have you run into anything like this lately? How do you navigate the tech side of looking out for your loved ones?

Asking out loud because I don't have all the answers (and because many minds working together are better than one). What I do know is that being unafraid to ask the hard questions is our collective superpower and greatest strength.