Open Source, Agents, and Specialization: What's Next in AI?

I spend my days building generative AI software and partnering with teams who push these systems into real-world workflows. Over the last few years I’ve watched an important shift: the conversation has moved from “one big model to rule everything” toward a much more practical and exciting place—open source foundations, specialized models, and agents that work together and with people.

I recently sat down with Constantine Bueller, a longtime AI engineer turned investor, to compare notes about where things are headed. We covered what enterprises care about, the technical gaps that must be closed for agents to scale, and how founders should think about building and evaluating systems in this new era. I want to summarize the practical ideas that came out of that conversation and expand on them, so you can walk away with a clear sense of what to prioritize if you’re designing, adopting, or investing in AI systems today.

🌐 Why open source matters right now

Open source has moved from an ideological preference to a pragmatic necessity in many settings. There are three clear reasons why open source foundations matter for enterprises and startups alike:

Control and transparency: enterprises want to own their data and understand model behavior. When you can inspect and fine tune model weights, it’s much easier to integrate AI into regulated, security-sensitive workflows.
Adaptability: open models enable faster specialization. Teams can distill, compress, and re-architect base models to suit specific latency, cost, and accuracy trade-offs rather than being forced into a one-size-fits-all API.
Community-driven standards: shared building blocks—models, libraries, and protocols—accelerate interoperability and help the ecosystem converge on effective patterns.

"Open source played a key role in adoption because it's a lot easier to take an AI into your enterprise when you actually control the weights," Constantine said. "It gives more power to the enterprise and ultimately to the consumer."

There is another practical point here. Open source lowers the friction for experimentation. When teams can run experiments locally, iterate on architectures, and push updates into private environments, they get to a minimum viable product much sooner. That shortens feedback loops and reduces the risk of costly integration surprises later on.

🤖 From large foundational models to specialized agents

We have moved through several interlocking phases over the last few years. First, large language models showed what was possible for text generation. Then systems started to be bolted into workflows using retrieval augmented generation, or RAG. Now the industry is pivoting to agents—software entities that can reason, take actions, and collaborate with other agents or humans.

That transition is not arbitrary. Agents are the natural next step for three reasons:

They encapsulate behavior: An agent is not just a model. It is a model plus memory, tool access, policies, and an interface for human oversight.
They can specialize: Instead of trying to make a single model handle every domain, teams deploy many agents specialized for specific tasks—scheduling, sales outreach, legal research—each optimized for accuracy and verifiability in its niche.
They compose: Agents can communicate and coordinate, allowing complex workflows to be split into smaller, tractable pieces that are easier to evaluate and improve.

Startups are already thinking beyond solo agents. Constantine described how many teams are prototyping swarms of agents that coordinate, hand off subtasks to one another, and run background processes to automate repeatable work. That’s the beginning of what I call the agent economy, or the market of digital workers that augment human teams.

🔑 The three technical challenges to unlock agents

If agents are going to become a fundamental part of software stacks, three engineering problems must be solved at scale: memory, communication protocols, and AI security. Each is tricky in its own way, and the solutions will shape the architectural contours of future systems.

Memory

An agent without reliable memory is like a colleague who forgets who you are every time you meet. There are two memory dimensions worth distinguishing.

Memory about users and context: RAG and vector stores are already used to attach context to queries—customer histories, internal documents, past tickets. That’s useful for making responses relevant, and it will remain a core technique.
Memory about the agent itself: This is the harder problem. How does an agent maintain consistency in its own persona and knowledge over time? If an agent’s behavior changes unpredictably each session, it loses trust. We need mechanisms for persistent identity, stable policy representations, and efficient ways to update model state without retraining from scratch.

These two forms of memory map to different engineering constructs: some memory should be treated like persistent storage (large, slow updates), and some should be like RAM or cache for the current dialog. The right mix will depend on application requirements.

Communication protocols

Agents need a lingua franca. When you have systems from multiple vendors, each exposing different capabilities and tool interfaces, coordination breaks down without shared standards. This is one area where open source is essential: common protocols are what let agents call each other’s services, negotiate responsibilities, and synchronize state. Think of it as an evolution comparable to how TCP/IP enabled the internet. A common communication layer reduces friction and accelerates cross-vendor innovation.

AI security

Security changes its shape in an agentified world. Agents that hold memories, take actions, and communicate with other agents create new attack surfaces. The scale and nature of threats will change.

One useful mental model is to invert the physical world assumptions: there is no reason to replicate the human security guard model where one defender protects many. Instead, we might end up with many security agents protecting a single cognitive resource. That changes both architecture and cost allocation. It also means we must build detection, verification, and containment capabilities into agent runtimes so that malicious behaviors can be caught and mitigated automatically.

🧪 Evaluation and the stochastic mindset

Classical software engineering and many traditional ML systems assume determinism. You give the system inputs and you expect consistent outputs. Generative models are inherently stochastic. That shift affects how we design experiments, measure success, and move systems into production.

Here’s what that means in practice:

Expect distributions, not single answers: A model might produce different outputs for the same prompt at different times. This variation is a feature of the way these models generalize, not necessarily a bug.
Design verifiers: For many tasks you must layer a verifier on top of the generative model. For code generation, the verifier is simple: does the code compile and pass unit tests? For high-stakes domains like medicine, verification might require domain experts—and that is expensive.
Shift from academic benchmarks to real-world evals: Benchmarks are helpful early on, but they don’t always reflect production risk. Real-world leaderboards and competitions can be more telling. Constantine shared an example of a startup that registered an AI on HackerOne and rapidly rose to the top of the vulnerability-finding leaderboard. That sort of outcome signals real-world capability faster than many academic metrics.

To bridge the verification gap, the industry needs better synthetic data generation, richer simulated environments or gyms for training, and specialized RL environments that reflect real task distributions. The more ways we can simulate or approximate the real world cheaply, the faster we can iterate on specialized agents.

🏭 Specialization, distillation, and the enterprise trade-offs

Enterprises want accuracy, privacy, and low cost of ownership. That combination pushes teams toward specialization and distillation—techniques to take a large, general-purpose model and compress or fine tune it into a smaller model tailored for a specific domain.

There are a few forces at work here:

Cost and latency: Running a massive model for every query is expensive. Distilled or domain-specific models can hit the same or better accuracy at a fraction of the compute and with lower latency.
Privacy and compliance: Enterprises often cannot send sensitive IP out to a third-party API. Hosting a distilled model on premises or in a controlled cloud environment keeps the data within governance constraints.
Optimization and lifecycle: Model development is becoming indistinguishable from software development. Teams retrain, re-evaluate, and release incremental improvements. Optimizing a model for inference costs and runtime performance is as important as adding new capabilities.

I like to demystify the process by leaning on a software development analogy: you collect data, train the model, validate with tests, deploy, monitor, and iterate. No shipped model is ever “done.” It accumulates feature requests, bug reports, and performance regressions. This lifecycle thinking makes the stakes and the expected work more realistic to organizations used to traditional software engineering.

📈 Investing up the stack: why the agent layer is hot

Investors and builders are shifting attention away from the base model alone and toward the app layer—the systems that connect models to people and processes. There are several reasons:

Because adoption is a people problem: AI must help people do their jobs better, not replace them. Systems that respect human workflows and amplify expert performance often produce much higher ROI.
Because composability creates value: The agent layer stitches models, tools, verifiers, and interfaces together, producing outcomes that single models cannot achieve on their own.
Because trust matters: Applications that offer effective human-in-the-loop controls dramatically increase adoption and reduce risk.

Let me give a concrete illustration. I work with a company that built an agent operating system for revenue teams. They started by automating simple tasks like drafting sales emails and following up with prospects. Initially some teams allowed the agent to auto-send messages, but conversion rates were low.

When the product changed to keep the seller in the loop—letting the agent draft the email but requiring the human to edit and approve before send—response rates tripled. That is a powerful lesson: integrating humans into the loop, not removing them, often produces the best outcomes.

🌍 World models and the next wave of innovation

We have been moving toward richer data modalities over time: from numbers to text, then to images, voice, and video. Each step increases the amount and fidelity of information flowing into a system. The next major frontier is world models—dense, continuous representations of the physical and digital environment that an agent can reason about and act within.

World models are extremely data intensive. They combine sensor streams, spatial understanding, temporal sequencing, and causal reasoning. When we get them right, they become the foundation for robotics, autonomous systems, and complex simulation-driven decision making.

Why do world models matter?

They allow long-term planning: A world model gives agents the ability to simulate outcomes and choose sequences of actions rather than reacting one step at a time.
They enable safe physical interaction: Robots and autonomous systems need robust models of the world to operate safely in human environments.
They unlock large scale automation: Tasks that require observing, predicting, and acting over long time horizons—logistics, manufacturing, and some forms of healthcare—become tractable with a world model layer.

I believe world models will be the base layer for the next big wave of robotic and autonomy applications. That’s an area where investments in both compute and data infrastructure will pay off massively.

⚙️ Advice to founders and enterprise leaders

If I could give one succinct piece of advice to founders, it’s to iterate like a stochastic gradient algorithm. Gather a batch of information, update, and repeat. You will never have perfect knowledge because much of the signal you need lives in the heads of your customers. So design processes that let you learn quickly and update confidently.

Here are practical maxims I encourage founders to follow:

Pick a worthy mountain: Choose a problem that’s big enough to matter. Small features may be quick to ship but rarely produce transformational business value.
Optimize for feedback velocity: Build verifiers and tests that let you evaluate whether your system is improving. For code generation, that means unit tests; for other domains, synthetic data and domain-specific signals become your unit tests.
Design for human-in-the-loop: Start with humans as validators and incorporate automation incrementally. Often the best path is to give experts better tools so they perform faster and more accurately.
Embrace specialization: Don’t try to make your model do everything. A focused agent for a specialized use case generally beats a generalized model in terms of accuracy and cost.
Invest in evaluation infrastructure: Build or adopt gyms, simulators, and verifiers that reflect real-world distributions for your domain. That investment pays dividends in deployment speed and reliability.

Enterprise leaders should recognize that adoption happens on a different cadence. Legacy systems and compliance constraints slow the pace. But the path to successful adoption is getting clearer: fine tune where needed, distill for production cost targets, and layer in human oversight to build trust.

✅ A practical checklist for adopting open models and agents

Here is a compact checklist I use when advising teams on product and architecture decisions. It’s practical, and you can apply it whether you are a founder, product manager, or engineering lead.

Define the verifier: How will you measure correctness? Make the verifier as automated and objective as possible.
Choose the right base model: Start with an open foundation that supports the level of control you need—weights, fine tuning, and architecture changes.
Plan for memory tiers: Separate short-term session memory from long-term identity or policy memory and design storage and access patterns accordingly.
Design communication protocols: If you expect multiple agents or services to interact, adopt or help define open communication standards early.
Build human-in-loop workflows: Determine where human verification is necessary and where automation can be safely applied.
Invest in synthetic data and gyms: Create or source environments to accelerate RL and fine-tuning for domain-specific skills.
Plan for security: Threat models change when agents have persistent identities and networked capabilities. Include detection, isolation, and recovery mechanisms.
Optimize for deployment: Distill models and tune inference stacks for cost and latency constraints. Treat model releases like software releases.

✍️ Closing thoughts

When I think about where AI is headed, I keep coming back to two themes: people and specialization. Technology is most valuable when it amplifies human expertise and fits comfortably into existing workflows. Building systems that respect that truth—by prioritizing verifiability, human-in-the-loop design, and specialization—will be the fastest path to adoption.

Open source will play a central role in this transition. It accelerates experimentation, enables enterprises to retain control, and provides fertile ground for shared standards that let agents communicate and coordinate. But open source is a means, not an end. What matters most are the systems we build with it—agents that remember, talk securely, and integrate into people’s daily work.

If you are building or adopting AI today, focus on the three unlocks—memory, communication, and security—design for verifiable outcomes, and iterate quickly with users in the loop. The next big wave, rooted in world models and rich multi-modal understanding, will be profound. But the companies that win it will be the ones that master the last mile: making these systems dependable, useful, and tightly integrated into human workflows.