DZone Spotlight

A few months ago, I saw something that made me rethink what coding assistants are actually capable of. A teammate was dealing with a frustrating race condition hidden deep inside a legacy service. It wasn't an obvious bug, and it had already taken quite a bit of time to investigate. Instead of digging through the code manually again, he described the problem to a coding agent, started it, and stepped away for a coffee. A few minutes later, the tests were passing, and there was a pull request ready for review. That was the moment it clicked for me. This wasn't just smarter autocomplete or a chatbot generating snippets. The agent had explored the codebase, identified the relevant files, tested different approaches, verified the fix, and completed the task with very little guidance. If you're still thinking about coding assistants the way we did a year or two ago, it's probably time to look at them differently. They're no longer just helping us write code. They're starting to take ownership of well-defined engineering tasks, and that's changing how many of us work. Autocomplete Was Never the Interesting Part For a long time, coding assistants were basically smarter autocomplete tools. They could suggest the next few lines of code, generate boilerplate, or explain an error message when you paste it into a chat window. Tools like GitHub Copilot made writing code faster, and there was no doubt that they improved day-to-day productivity. But you were still driving the entire process. You decided where to start, which files to open, what needed to change, and whether the generated code actually solved the problem. The assistant helped you write code, but it wasn't responsible for the task itself. That's what has changed over the last year. Today's coding agents are built to work toward an objective rather than simply respond to prompts. Give them a goal, access to your project, a terminal, and permission to run tests, and they'll start working through the problem on their own. They inspect the codebase, identify the relevant files, make changes, run the tests, learn from the results, and keep iterating until they either solve the problem or genuinely need your input. They're no longer trying to predict your next line of code. They're trying to complete the task. A typical workflow looks something like this: Shell $ agent run "checkout_test.py is flaky, find out why and fix it" ▸ Reading checkout_test.py, checkout_service.py, order_state.py ▸ Running test suite 5x to reproduce flakiness... 2/5 failures ▸ Hypothesis: race condition between inventory lock release and payment webhook handler ▸ Reading inventory_lock.py, webhook_handler.py ▸ Adding targeted logging, re-running... ▸ Confirmed: webhook can fire before lock release commits ▸ Editing inventory_lock.py — moving lock release after commit ack ▸ Running full test suite 10x... 10/10 pass ▸ Opening PR #4127: "Fix race condition in checkout lock release" Done in 6m 42s. 1 file changed, 9 insertions, 3 deletions. $ agent run "checkout_test.py is flaky, find out why and fix it" ▸ Reading checkout_test.py, checkout_service.py, order_state.py ▸ Running test suite 5x to reproduce flakiness... 2/5 failures ▸ Hypothesis: race condition between inventory lock release and payment webhook handler ▸ Reading inventory_lock.py, webhook_handler.py ▸ Adding targeted logging, re-running... ▸ Confirmed: webhook can fire before lock release commits ▸ Editing inventory_lock.py — moving lock release after commit ack ▸ Running full test suite 10x... 10/10 pass ▸ Opening PR #4127: "Fix race condition in checkout lock release" Done in 6m 42s. 1 file changed, 9 insertions, 3 deletions. Nobody told me which file the bug was in. That's the part that used to be the job. Where This Actually Helps (And Where It Doesn't) Let's be realistic. These tools aren't writing every line of code for us, and they probably shouldn't. What I've noticed is that developers are becoming much more selective about what they hand over. Tasks that are repetitive and easy to verify are usually fair game. Things like fixing flaky tests, updating dependencies, generating CRUD code, analyzing logs, or tracking down why an endpoint is suddenly running slower than expected. On the other hand, work that involves architecture decisions, business logic, security, or long-term design still benefits from human judgment. Those are the areas where context matters, and where a conversation often leads to a better outcome than simply asking an agent to take over. In practice, the most productive teams aren't trying to replace developers. They're using these tools to take care of the repetitive work, leaving engineers with more time to focus on solving the problems that actually require experience and critical thinking. One thing I've learned is that the size of a task doesn't really determine whether it's a good candidate for delegation. What matters more is how easy it is to verify the result. For example, a large refactor across a codebase you know well can be a good fit because you can review the changes, run the tests, and quickly spot anything that looks wrong. On the other hand, a tiny change in something like a payment reconciliation flow might deserve far more attention. Even if it's only a few lines of code, the impact of getting it wrong can be significant, and it's not always easy to validate the outcome with a quick review. In other words, I don't decide based on how much code is involved. I decide based on how confident I can be that the result is correct. Another change that doesn't get talked about as much is how these agents are being guided. In the beginning, everything depended on prompts. Every new session meant explaining your project's structure, coding standards, and the little rules your team follows. That's becoming less common. Most modern coding agents now look for project-level configuration files before they start making changes. These files capture things like coding conventions, architectural guidelines, testing requirements, and simple rules such as "don't modify the migrations folder without approval." The benefit is obvious. Instead of repeating the same instructions every time, you define them once and let the agent follow them consistently across sessions. It's a small change on the surface, but it makes these tools feel much more like a teammate who's familiar with your project instead of someone who needs the same onboarding every single day. Shell # Project conventions for AI agents - Run `pnpm test` before opening any PR, not `npm test` - Never modify files under /migrations directly — generate a new migration instead - API responses must match the schema in /schemas, run `pnpm validate:schema` after changes - Prefer editing existing utility functions in /lib/utils over creating new ones - Ask before adding a new npm dependency That configuration file does much more than provide instructions to the agent. It captures the small details that every team relies on but rarely documents well. Things like coding conventions, preferred workflows, and project-specific rules that usually exist only in the minds of experienced engineers or are buried somewhere in an old wiki that hardly anyone opens. By putting that knowledge into a single place, every coding agent starts with the same understanding of the project instead of having to learn those rules from scratch every time. The Multi-Agent Thing Is Real, Not Just Marketing Another trend that's becoming hard to ignore is multi-agent orchestration. Instead of relying on a single agent to handle everything, one agent acts as a coordinator and breaks the work into smaller, focused tasks. For example, one might handle the backend changes, another updates the frontend, while a third reviews the code for potential security issues. Once each task is complete, the coordinator brings everything together into a single result. I'll admit, I was skeptical when I first heard about this approach. It sounded like another buzzword that would look impressive in demos but struggle in real projects. But after seeing it work on practical tasks, like adding OAuth support without breaking an existing authentication flow, it started to make more sense. The work was naturally divided into backend changes, frontend updates, and a security review, with each part progressing at the same time instead of waiting for the previous step to finish. It's not the right solution for every problem, but for tasks that can be split into independent pieces, it can save a surprising amount of time. Shell $ agent run "add OAuth login with Google, keep existing email/password flow working" ▸ Planning: 3 subtasks identified ├─ [backend] OAuth token exchange + session handling ├─ [frontend] Login button + redirect flow └─ [security] Review token storage, CSRF handling ▸ Dispatching subtasks (parallel)... [backend] editing auth_service.py, session_store.py [frontend] editing LoginPage.tsx, auth_client.ts [security] reviewing diffs as they land ▸ [security] flagged: refresh token stored in localStorage, recommend httpOnly cookie instead ▸ [backend] applying fix — switching to httpOnly cookie storage ▸ All subtasks complete, running integration tests... 47/47 pass Like any new approach, it isn't perfect. Coordinating multiple agents adds its own complexity, and there are plenty of situations where a single, well-configured agent is still the better choice. For tasks that require careful reasoning or involve lots of dependencies, keeping everything in one place is often simpler and more reliable. Where multi-agent workflows really shine is when the work can be divided into independent pieces. Backend, frontend, testing, and security reviews can all move forward at the same time instead of waiting on one another. It's not a silver bullet, and it won't replace every workflow. But when the problem fits the approach, the productivity gains can be surprisingly real. The Bill Comes Due Somewhere Of course, there are trade-offs. As these agents become more capable, they're also becoming more expensive to run. Longer sessions, larger context windows, and frequent tool calls can increase costs much faster than many teams expect. I've noticed that the conversation is slowly shifting. Instead of asking, "Is this the fastest agent?" teams are starting to ask, "Is it worth the cost?" That means looking beyond impressive demos and measuring things that actually matter, like the cost of resolving an issue, completing a feature, or reviewing a pull request. Performance is still important, but it's no longer the only metric. Finding the right balance between capability, speed, and cost is becoming just as important. Security is another area that deserves more attention. The same capabilities that allow an agent to review code, identify vulnerabilities, or strengthen an authentication flow can also be misused if the wrong person has access to those tools. That doesn't mean these agents are unsafe, and it certainly isn't a reason to avoid them. It simply means they should be treated like any other powerful engineering tool. If an agent has access to your terminal, repository, or production environment, those permissions need to be managed carefully. Giving an agent unrestricted shell access without proper controls isn't very different from giving a new team member broad access on their first day. As these tools become part of everyday development, security, access control, and auditing need to be considered from the beginning, not added later as an afterthought. So What Actually Changes for You If you're thinking about adding one of these tools to your development workflow, or your team has already started using them, but you're still trying to understand where they fit, here are a few lessons I've picked up along the way. Here are a few things that have stood out to me while working with these tools. Look beyond the model. Two coding agents can use the same underlying model and still deliver very different results. What often makes the biggest difference is how they manage context, permissions, available tools, and how they recover when something goes wrong. Don't choose a tool based only on the model it advertises.Start with low-risk tasks. Let the agent handle work that's easy to review and validate, like fixing flaky tests, updating dependencies, writing migration scripts, or investigating logs. As your confidence grows, you can gradually trust it with more complex work.Document your project's conventions. A simple configuration file that explains coding standards, testing requirements, and project-specific rules can save a lot of time. It helps the agent understand your project from the beginning instead of learning the same lessons in every session.Keep an eye on cost. Longer sessions, repeated tool calls, and large context windows can add up quickly. It's worth monitoring how much each task costs so you can balance productivity with efficiency instead of being surprised by your monthly bill. I don't believe developers are being replaced. What I do think is changing is how we spend our time. Writing code is becoming faster, but reviewing changes, making architectural decisions, understanding business requirements, and ensuring quality are becoming even more important. In many ways, developers are moving from writing every line of code to guiding the overall process. We define the problem, review the solution, make the final decisions, and step in whenever judgment or experience is needed. That's a different way of working, and we're still figuring out what it looks like in practice. Whether it's ultimately a better way to build software is something only time will answer. But one thing feels clear already: the role of a software engineer is evolving, and learning how to work effectively with these tools is becoming an important part of the job.

Your last pentest is already out of date. The moment you shipped new code after that report, your risk profile changed, and nobody re-tested it. That's the reality most teams are living in. Nearly 29% of organizations still lack continuous vulnerability monitoring, relying instead on periodic scans that miss threats attackers are actively exploiting right now. Annual testing made sense when releases happened twice a year. It doesn't anymore. Your CI/CD pipeline ships changes weekly, your APIs multiply monthly, and your attack surface never stops moving. This piece breaks down why continuous testing has shifted from a security team's wish list to a business requirement, and what it actually takes to build one that works. Understanding Continuous Application Security Testing Continuous application security testing is the practice of running automated security checks every time your code changes. Instead of waiting for a scheduled quarterly review, you should scan your pipelines during daily deployments. This approach ensures that no code goes live without a fast security check. According to the Black Duck BSIMM16 report, high-performing engineering teams integrate real-time vulnerability detection directly into their CI/CD pipelines. This process relies on an automated security testing approach to discover software flaws instantly. It shifts security from a periodic event to a constant, background process. The main focus of this approach is continuous testing with exploit validation. It catches critical flaws like broken object-level authorization before attackers can exploit them. By checking your attack surface daily, you protect live applications without forcing your development team to slow down. Why Traditional Security Testing Is No Longer Enough Traditional security testing fails because modern software development moves too fast. Legacy assessment methods like quarterly scans cannot keep up with rapid deployment cycles, leaving application endpoints exposed to real-world threats. Rapid Deployment Cycles Break Periodic Schedules Modern development teams ship code changes daily or hourly. A scheduled security test only captures a single moment in time. The very next code push can introduce critical flaws, making a recent assessment report completely obsolete. The Exploit Window is Shrinking Rapidly Attackers utilize advanced solutions to scan vulnerabilities immediately after discovery. According to recent Cybersecurity and Infrastructure Security Agency (CISA) reports, threat actors target new flaws within hours. Waiting months for a security scan leaves a massive window open. Modern App Architectures Increase Attack Surfaces Applications rely heavily on cloud APIs, microservices, and micro-frontend structures. This creates complex data paths that static legacy testing cannot map. Without continuous verification, broken access controls and hidden data leaks go completely unnoticed inside these sprawling networks. Compliance Audits Fail to Prevent Attacks Passing a standard compliance audit does not guarantee active protection. Regulatory checks often focus on documentation and basic patch levels rather than real-world exploit validation. A system can achieve compliance while remaining completely vulnerable to active web exploits. High False Positive Rates Drain Engineering Resources Legacy security testing methods often produce massive lists of unverified bugs. Security teams waste hours manually filtering out false alarms. This friction slows down software delivery and causes friction between development groups and security personnel. The Biggest Risks of Not Testing Continuously Skipping real-time security reviews exposes production code to critical software vulnerabilities. Without continuous validation, hidden entry points and data security gaps remain open for attackers to exploit. Accumulating severe security debt: Untested code builds up flaws over time. This makes future remediation complex and highly expensive for engineering teams to resolve. Exploited broken object-level authorization: Attackers target unverified API endpoints easily. They manipulate object identifiers to gain unauthorized access to sensitive user data. Silent third-party dependency exploits: Open-source libraries introduce hidden bugs regularly. Without real-time dependency scanning, malicious updates can compromise your entire software supply chain unnoticed. Unchecked web application misconfigurations: S3 buckets and access control rules get altered during fast updates. These small changes expose critical databases to the public web. Extended attacker dwell time: Threat actors slip into quiet system gaps easily. They steal data for months before periodic testing cycles finally flag the breach. Costly emergency patch deployments: Discovering severe flaws right before an audit forces rushed fixes. This disrupts product roadmaps and introduces unstable code into production systems. Compliance failure and financial penalties: Lacking continuous monitoring violates modern data privacy mandates. This leads to failed security audits and heavy regulatory fines for your business. Business Value of Moving to a Continuous Security Model Transitioning to continuous security safeguards critical digital assets while optimizing engineering speed. Real-time exploit validation protects corporate reputation, ensures regulatory compliance, and reduces the financial impact of data breaches. Lower Remediation and Engineering Costs Fixing software vulnerabilities early in the development lifecycle is significantly cheaper. Continuous validation prevents security flaws from reaching production. This eliminates the need for expensive emergency hotfixes and saves valuable developer hours. Faster Secure Software Delivery Integrating security directly into CI/CD pipelines eliminates late-stage deployment bottlenecks. Engineering teams ship functional updates with confidence, knowing automated scans run in the background. Security becomes an accelerator rather than a roadblock. Frictionless Compliance and Audit Readiness Continuous monitoring maintains a constant state of compliance with frameworks like GDPR and PCI DSS. Instead of scrambling before annual security audits, organizations retain historical evidence of active threat management. Reduced True Positive Alert Fatigue Advanced testing platforms prioritize real exploit validation over theoretical bug lists. Filtering out false positives ensures security operations center teams only focus on validated threats. This focus optimizes overall incident response efficiency. Stronger Customer Trust and Brand Equity Demonstrating proactive data protection builds deep trust with enterprise clients. Continuous application testing proves your organization prioritizes information security. This competitive advantage helps accelerate sales cycles and protects brand reputation. Best Practices for Implementing Continuous Application Security Testing Deploying continuous security requires blending automation seamlessly into existing developer workflows. Following industry blueprints ensures real-time exploit validation keeps application platforms secure without disrupting rapid software release cycles. Integrate scans directly into CI/CD pipelines: Embed automated security into your daily deployment pipeline. Running fast vulnerability checks on every code commit stops bugs before they reach production. Focus on live exploit validation: Prioritize an approach that actively tests whether a bug is truly exploitable or not. Confirming real attack paths eliminates time wasted on harmless false positives. Automate API endpoint discovery: Modern web apps shift constantly. Use a dynamic discovery approach to find hidden endpoints and protect against broken object-level authorization gaps automatically. Implement real-time dependency tracking: Scan open-source packages during every build cycle. This process flags vulnerable third-party libraries and protects your software supply chain instantly. Combine automation with strategic manual tests: Automated scanning handles repetitive code checking efficiently. Use manual penetration testing for complex business logic flaws that automated scanners miss. Enable MFA-aware testing flows: Ensure your testing software can bypass multifactor authentication securely. Authentic user journey testing reveals hidden security flaws inside deep, protected application layers. Train developers on remediation context: Provide engineering teams with clear exploit evidence right inside their dashboards. Detailed contextual reports help developers fix critical flaws quickly without extra friction. Wrapping Up Point-in-time testing was never designed for how software ships today. Weekly deployments, expanding APIs, and AI-generated code all move faster than a once-a-year security check can track. Continuous application security testing closes that gap. It catches vulnerabilities the moment they enter your codebase, validates real exploitability, and gives your team evidence, not guesswork, when auditors come asking. The organizations pulling ahead aren't the ones testing more often. They're the ones testing continuously, prioritizing exploitable risk over noise, and treating security as part of how they build, not an afterthought.

As Large Language Models (LLMs) become increasingly integrated into enterprise applications, optimizing response time and reducing operational costs have become critical priorities. One of the most effective techniques for achieving both is Prompt Caching. Instead of processing identical prompt segments repeatedly, prompt caching allows AI systems to reuse previously computed prompt representations, minimizing redundant computation. While tokenization converts text into tokens that the model understands, prompt caching goes a step further by reusing the processing of unchanged token sequences, resulting in faster inference, lower latency, and reduced API costs, especially in applications with repetitive system prompts or recurring contextual information. How Prompt Caching Works Think of prompt caching as a “memory shortcut” for AI models. Every prompt is first tokenized, but when the same prompt prefix appears again, the model doesn’t need to process those tokens from scratch. Instead, it retrieves the cached computation and only processes the new or modified portion of the prompt. How Prompt Caching Works This mechanism is particularly valuable in AI assistants, enterprise chatbots, coding copilots, document analysis platforms, and Retrieval-Augmented Generation (RAG) systems where a significant portion of the prompt remains unchanged across multiple requests. Best Practices to Maximize Prompt Cache Efficiency To fully leverage prompt caching, organizations should design prompts strategically. Keep system instructions consistent, place static context before dynamic user inputs, avoid unnecessary formatting changes, and modularize prompt templates. These practices increase cache hit rates, reducing both processing time and infrastructure costs. Monitoring cache performance metrics, such as cache hit ratio, latency improvements, and token savings, helps teams continuously optimize AI workloads while maintaining response quality. Business Benefits and Real-World Impact Prompt caching delivers measurable business value beyond technical optimization. Organizations can reduce AI inference costs, improve application responsiveness, support higher request volumes, and enhance the overall user experience. Development teams also benefit from more predictable performance and scalable AI architectures. As enterprise AI adoption grows, prompt caching is becoming an essential optimization technique for building efficient, reliable, and cost-effective generative AI solutions. Where Prompt Cache Is Stored: Understanding the Architecture Where a prompt cache is stored depends entirely on which level of the caching architecture you are referring to. To understand where it lives, it is helpful to divide prompt caching into its two primary forms: Provider-Native Caching (Model-Level) When you use built-in prompt caching features from providers such as OpenAI, Anthropic (Claude), Google (Gemini), or DeepSeek, the cache is managed internally within the provider’s cloud infrastructure. What is Stored The cache does not store text or responses. Instead, it stores KV Tensors (Key-Value pairs). These are the raw, mathematical attention states that the model's neural network calculated during the "prefill" phase of your prompt Where Will it Live? GPU VRAM / High-Speed RAM: Because these tensors must be accessed instantly to keep latency ultra-low, they are stored directly in the high-speed volatile memory (VRAM) of the AI chips (GPUs/TPUs) or ultra-fast host system memory in the provider's data centers. Internal Distributed Storage: Since GPU memory is highly constrained and expensive, providers use advanced, proprietary cache-eviction systems. If a cache prefix isn't used for a few minutes (the Time-to-Live or TTL), it is automatically evicted (deleted) from the GPU memory to make room for other users Who Has Access? The provider manages this entirely behind the scenes. You cannot download, inspect, or manually move these KV tensors; the system simply checks the memory automatically during your API call and applies a discount if it finds a match. Application-Level Caching (User-Controlled Layer) If you are building your own caching layer in front of the LLM API to save even more money by bypassing the LLM entirely for repeat queries, you get to choose where it is stored In-Memory Databases (Most Common) Platforms like Redis or Memcached are the industry standard. Because they store data directly in RAM, they can fetch cached prompts in microseconds Vector Databases (For Semantic Caching) If you want to detect "semantically similar" prompts (e.g., matching "How do I reset my password?" with "I forgot my password"), the cache stores the text embeddings. This is stored in vector databases like Pinecone, Milvus, Qdrant, Weaviate, or pgvector (PostgreSQL) Relational / NoSQL Databases (For Archive/Backup) Standard databases like MongoDB, DynamoDB, or PostgreSQL are used to persistently store historical prompt-response pairs, though they have slightly higher retrieval latency than Redis Building a Semantic Cache With Redis involves upgrading from traditional "exact-match" caching to vector-based similarity caching. Instead of storing raw text, you store the mathematical representation (embeddings) of prompts. When a new prompt comes in, you convert it to an embedding and ask Redis to find the "nearest neighbor" (most similar prompt). If the similarity score exceeds your defined threshold (e.g., 95% similar), it's a Cache Hit. Here is the step-by-step guide to building a semantic cache using Python, Redis Stack (which includes vector search), and an embedding model (like OpenAI's). Prerequisites Redis Stack: You must use Redis Stack (or Redis Enterprise), as standard Redis does not support vector search. You can run it locally via Docker: docker run -d -p 6379:6379 redis/redis-stack-server:latest. Python Libraries: Install the required clients. pip install redis openai numpy: Redis also has a dedicated library called redisvl (Redis Vector Library) built specifically for this, which abstracts a lot of the boilerplate. Note: Redis also has a dedicated library called redisvl (Redis Vector Library) built specifically for this, which abstracts a lot of the boilerplate. The workflow follows four steps: Embed: Convert the incoming user prompt into a vector embedding. Search: Query Redis using a K-Nearest Neighbors (KNN) vector search. Evaluate: If the highest similarity score is above your threshold (e.g., > 0.92), return the cached response. Fallback and store: If no match is found, send the prompt to the LLM, return the response to the user, and store the new embedding and response in Redis Conceptual Python Implementation How the logic flows using standard redis-py and OpenAI: Python import redis import numpy as np from openai import OpenAI from redis.commands.search.query import Query # 1. Initialize Clients redis_client = redis.Redis(host='localhost', port=6379, decode_responses=True) openai_client = OpenAI(api_key="YOUR_API_KEY") # Configuration THRESHOLD = 0.95 # 95% similarity required for a cache hit INDEX_NAME = "prompt_cache_idx" def get_embedding(text): """Convert text to an embedding vector.""" response = openai_client.embeddings.create( input=text, model="text-embedding-3-small" ) return np.array(response.data[0].embedding, dtype=np.float32).tobytes() def check_semantic_cache(prompt_text): """Search Redis for a semantically similar prompt.""" query_vector = get_embedding(prompt_text) # Construct a KNN Vector Search Query in Redis q = Query(f"*=>[KNN 1 @prompt_vector $vec AS score]")\ .return_fields("response", "score")\ .sort_by("score")\ .dialect(2) res = redis_client.ft(INDEX_NAME).search( q, query_params={"vec": query_vector} ) if res.docs: # Redis returns distance (0 is perfect match). Convert to similarity. similarity = 1 - float(res.docs[0].score) if similarity >= THRESHOLD: print(f"✅ Cache Hit! (Similarity: {similarity:.2f})") return res.docs[0].response print("❌ Cache Miss.") return None def store_in_cache(prompt_text, llm_response): """Store the new prompt and response in Redis.""" prompt_vector = get_embedding(prompt_text) # Store as a Redis Hash doc_id = f"cache:{hash(prompt_text)}" redis_client.hset(doc_id, mapping={ "prompt": prompt_text, "response": llm_response, "prompt_vector": prompt_vector }) # Optional: Set a Time-To-Live (TTL) so the cache clears old entries redis_client.expire(doc_id, 86400) # 24 hours Best Practices for Production Use a library: Instead of writing the raw vector math and RediSearch queries yourself, use RedisVL (pip install redisvl) or LangChain's Redis Cache integration. They have built-in SemanticCache classes that handle index creation and threshold tuning with just 3 lines of code. Tune your threshold carefully: A threshold that is too low (e.g., 0.80) will cause "false positives" (returning an answer to a question that is only vaguely related). A threshold too high (e.g., 0.99) defeats the purpose, acting almost like an exact-match cache. Test with 0.92 to 0.95 as a baseline. Filter by user/tenant: If you are building a multi-tenant app, make sure to add metadata tags (like user_id or tenant_id) to your Redis hashes. Your vector query must pre-filter by the user_id, so User A doesn't accidentally get a cached response meant for User B. Cost Savings by Major Provider LLM providers apply discounts specifically to input tokens that hit the cache (output tokens are always billed at the standard rate) Real-World Impact and Key Benchmarks Enterprise scale: One of the big Tech companies, like TikTok, has reported cutting their AI agent inference costs by 50% with minimal code adjustments. Agentic architectures: For complex, long-running agentic workflows (where a system prompt and conversation history are repeatedly sent over dozens of steps), prompt caching typically achieves 78% to 81% total cost reductions because the massive system instructions only need to be processed once. Break-even point: On platforms like Anthropic (which charge a 25% premium to write to the cache), you only need to hit the cache twice on a given prompt prefix to break even and start saving money. Every subsequent read is essentially 90% off. In addition to saving money, prompt caching dramatically improves user experience by skipping the heavy "prefill" computation. It reduces Time-to-First-Token (TTFT) by 50% to 85%, meaning long documents or extensive chat histories return responses in a fraction of a second instead of causing a noticeable delay. Take Action: Build Smarter AI Applications Prompt caching is no longer an optional optimization—it’s a competitive advantage for organizations deploying AI at scale. If you’re building enterprise AI applications, evaluate where repetitive prompts exist and redesign your prompt architecture to maximize cache utilization. Small changes in prompt design can lead to significant savings in cost, latency, and compute resources.

When engineering teams build distributed systems, they naturally reach for REST over HTTP/1.1 with JSON payloads. JSON is readable, universally supported, and trivially easy to debug with any browser or proxy tool. For early-stage services handling modest traffic, that convenience is a genuine engineering asset. But as microservice topologies scale toward hundreds of nodes handling tens of thousands of concurrent requests, text-based serialization frequently evolves from a minor convenience into a measurable architectural bottleneck. CPU utilization climbs, p99 latencies widen, and intra-zone bandwidth costs quietly compound across every internal service hop. Transitioning internal service-to-service communication to Protocol Buffers (Protobuf) over HTTP/2 via gRPC is one of the most effective and high-leverage responses to this problem. This article breaks down exactly why JSON degrades at scale, how Protobuf's binary wire format addresses those root causes, and how to execute a zero-downtime migration without breaking your running services. The Hidden Cost of Text-Based Serialization at Scale To understand why JSON degrades at high throughput, you have to look past network bandwidth and examine CPU behavior directly. JSON is a text-based, schema-less format. Every time a microservice ingests a JSON payload, the runtime must allocate memory on the heap, parse raw strings, map keys to internal structs via reflection, and convert values to their respective data types. At low volumes, this parsing overhead is negligible. At enterprise scale, it compounds into a real problem across two distinct dimensions. 1. CPU-Bound Allocation and GC Churn In languages with managed memory runtimes, such as Go, Java, and Node.js being the most common in microservice architectures, parsing thousands of large JSON strings per second causes significant garbage collection pressure. Each incoming payload generates a burst of short-lived string allocations on the heap. The garbage collector is forced to run more frequently to reclaim this memory, and in runtimes that use stop-the-world collection phases, this directly spikes p99 tail latencies. The problem is not that JSON parsing is intrinsically slow on a single call. The problem is that at scale, thousands of calls per second accumulate into sustained allocation pressure that the GC cannot absorb cleanly. 2. Network Payload Bloat JSON payloads are structurally verbose because every single message must explicitly include field names as strings. Consider this representative internal service message: JSON { "transaction_id": "tx_9988112233", "account_status": "ACTIVE", "retry_count": 3 } On the wire, this payload consumes roughly 85 bytes. More than half of those bytes (over 50) are dedicated purely to transmitting key metadata: the strings "transaction_id", "account_status", and "retry_count". These keys carry no runtime information that the receiving service doesn't already know from its own code. They are structural overhead repeated on every single message. Multiply this across millions of internal RPC calls through a service mesh and you are looking at gigabytes of redundant key data transmitted intra-zone every day. That's bandwidth you are paying for and CPU cycles you are spending to parse, without gaining any informational value. The Mechanics of the Binary Shift: Why Protobuf Moves the Needle Protocol Buffers eliminate text overhead by relying on a strict Interface Definition Language (IDL) and a highly compressed binary wire format. Instead of transmitting field names, Protobuf assigns each field a unique integer tag. When a message is serialized, the keys are stripped out entirely. The wire representation of any field is just its integer tag combined with a wire type identifier, followed by the raw data bytes. The equivalent of the JSON example above looks like this as a .proto definition: ProtoBuf syntax = "proto3"; message AccountTransaction { string transaction_id = 1; string account_status = 2; int32 retry_count = 3; } The same AccountTransaction message with the values tx_9988112233, ACTIVE, and 3 serializes to approximately 24 bytes on the wire — a reduction of roughly 72% compared to the JSON equivalent. Varints and Length-Delimited Encoding Two specific encoding techniques drive most of that size reduction. Varints (Variable-Length Quantities): Standard integers occupy a fixed 4 or 8 bytes regardless of their actual value. Protobuf varints use the most significant bit as a continuation flag, meaning small integers consume fewer bytes than large ones. The value 3 in the retry_count field above occupies exactly one byte on the wire. For the high-frequency small counters and status codes typical in microservice messages, this is a consistent win. Length-delimited encoding: Strings and nested messages are encoded with an explicit byte-length prefix followed by the raw byte block. The parser reads the tag, reads the length, and copies the exact memory block directly. There is no tokenization, no string-splitting, and no key-to-field mapping via reflection. This direct memory copy approach is what makes Protobuf deserialization significantly faster than JSON parsing in practice. Benchmarks from the go_serialization_benchmarks project (available on GitHub) consistently show Protobuf outperforming standard library JSON by 4–8x in throughput on typical message shapes. Architectural Trade-Offs: When to Move and When to Wait Migrating to Protobuf is not a universal improvement. It introduces distinct operational trade-offs that teams should evaluate honestly before committing. MetricJSON over HTTP/1.1Protobuf over HTTP/2 (gRPC)Human readabilityNative — clear text in proxy logsRequires compiled schemas or tooling like grpc-curl or protoscope to inspectSchema enforcementOptional — JSON Schema is separate from the formatMandatory — enforced at build time via protoc compilationNetwork efficiencyLow — verbose string keys on every messageHigh — packed binary tag-value pairs, no key transmissionCPU utilizationHigh — heap allocation, reflection, and string parsingLow — direct memory copies and varint arithmeticDebugging overheadLow — any HTTP tool worksHigher — binary streams require schema-aware toolingSchema registry costNone — ad hoc contract managementReal — .proto files must be versioned and distributed across teams The debugging and schema-management costs deserve emphasis because they are frequently underestimated. In a JSON-based system, any engineer can inspect a live request in a proxy log or with curl. In a Protobuf system, you need the compiled schema available to decode what is on the wire. Teams that invest in a proper schema registry and standardize on tools like grpcurl absorb this cost smoothly. Teams that don't will find debugging production issues significantly harder. The Edge vs. Mesh Topology Split The most pragmatic migration approach keeps JSON at the public API boundary while adopting Protobuf exclusively for internal service-to-service traffic. The API Gateway acts as the translation layer: it terminates public-facing REST/JSON requests from browsers and mobile clients, validates the incoming payloads, and transforms them into strongly-typed Protobuf messages before routing them across the internal service mesh. Public consumers never see binary formats. Internal services get the full efficiency benefit. This topology preserves external interoperability while capturing the performance gains where they matter most, which is inside the mesh, where requests fan out across many hops. Executing a Zero-Downtime Migration The core challenge in any serialization migration is that you cannot atomically redeploy every service simultaneously. Services must continue communicating during the transition. The following phased approach handles this safely. Phase 1: Dual-Stack Services Update each internal service to accept both JSON and Protobuf requests simultaneously, using the Content-Type header to distinguish them (application/json vs. application/x-protobuf). This is the strangler fig pattern applied to serialization. No existing traffic breaks, and you can validate Protobuf behavior against live traffic without fully cutting over. Phase 2: Canary Routing Once dual-stack services are deployed, route a small percentage of internal traffic, start with 1–5%, to the Protobuf path. Monitor p99 latency, error rates, and deserialization failure metrics at the canary boundary. This is the moment where schema mismatches and field mapping errors surface, and it is far better to find them at 1% traffic than at 100%. Phase 3: Full Cutover and JSON Deprecation After the canary validates correctly over a sufficient observation window (typically one to two release cycles), shift all internal traffic to Protobuf. Maintain the JSON code path for a deprecation period to support any lagging consumers, then remove it once all services confirm clean Protobuf-only communication. Mapping JSON Structures to Proto3 When moving from a schema-less JSON environment to a typed Proto3 environment, data structures need explicit definition. Here are the most common mapping decisions. Primitive and Complex Types Numbers: Map floating-point values to double or float. Map integers to int32, int64, or uint32. If values can be negative and small (common for status codes or offsets), use sint32 or sint64, which apply ZigZag encoding to make negative varints more compact.Arrays: Represent repeated values with the repeated keyword.Maps: Use the native map<string, string> syntax. Note that map fields cannot be marked as repeated. Bootstrapping Proto Definitions From Existing Payloads When you are migrating an existing system with dozens or hundreds of active message models, writing .proto definitions by hand from legacy JSON schemas is tedious and error-prone, especially when the source payloads contain deeply nested objects, polymorphic arrays, or inconsistent field naming conventions. A practical shortcut during the early scaffolding phase is to use a JSON-to-Protobuf converter utility. You feed in a representative sample payload, and it generates a baseline .proto definition that matches the field names, infers appropriate types, and assigns initial field numbers. The output is not final. You will still need to review type choices, apply sint32/sint64 where appropriate, and add optional markers for nullable fields, but it eliminates the mechanical first pass and lets engineers focus on the decisions that actually require judgment. This is particularly useful when onboarding a new team member to the migration or when tackling a legacy service whose JSON schema was never formally documented. Handling the Absence of Native Nulls Proto3 does not have a native null state for primitive types. Unset fields default to their zero value — empty string "" for strings, 0 for integers. In systems where an unset field and a zero-value field carry different semantic meaning, this distinction matters. Two approaches address this. The first is the optional keyword, which wraps the primitive in a field-presence tracker that lets the receiver distinguish "this field was not set" from "this field was set to zero": ProtoBuf syntax = "proto3"; message PaymentRecord { string payment_id = 1; optional int32 discount_percentage = 2; // Distinguishes "no discount" from "0% discount" } The second is Google's well-known wrapper types, which provide nullable primitives at the cost of a more verbose message structure: ProtoBuf import "google/protobuf/wrappers.proto"; message ExtendedTransaction { string id = 1; google.protobuf.StringValue middle_initial = 2; // Nullable string } For most use cases, optional is the cleaner choice. Wrapper types are useful when you need to nest nullable primitives inside repeated fields or maps. Managing Schema Evolution Without Breaking Running Services In a distributed environment with independent deployment cycles, schema changes are inevitable and dangerous if handled carelessly. Protobuf addresses this through strict backward and forward compatibility rules, but only if you respect two absolute constraints. Never change field numbers. The binary parser maps incoming bytes to fields purely by tag integer. If you change a field number on a deployed message, existing services will misread the data silently and without error. Never change the wire type for an existing tag. If a field needs to change from int32 to string, you must deprecate the old tag and introduce a new field with a new field number. Beyond those hard rules, backward compatibility allows you to add new fields freely. A service that receives a message with an unknown field number will simply ignore it. This means services can be updated independently and out of order without breaking communication, which is a critical property in a rolling deployment environment. Graceful Deprecation in Practice When phasing out an existing field, mark it with the deprecated option rather than deleting it. This preserves binary compatibility for services still reading the field while alerting downstream teams through compiler warnings: ProtoBuf message UserContext { string user_id = 1; string legacy_token = 2 [deprecated = true]; // Superseded by session_hash; remove after Q3 cutover string session_hash = 3; } Do not reuse the field number after deprecation. Reserve it explicitly using the reserved keyword to prevent future developers from accidentally reusing a tag that old binary data may still contain: ProtoBuf message UserContext { reserved 2; reserved "legacy_token"; string user_id = 1; string session_hash = 3; } Concrete Implementation: Deserializing Protobuf in Go The following example shows a typical internal Go service handler receiving and deserializing a Protobuf message using the current v2 API (google.golang.org/protobuf/proto). Note: the v1 package (github.com/golang/protobuf) is archived and should not be used in new code. Go package main import ( "fmt" "log" "time" "google.golang.org/protobuf/proto" pb "path/to/generated/pb" // Pre-compiled .pb.go output from protoc ) func processPayload(rawBytes []byte) (*pb.AccountTransaction, error) { transaction := &pb.AccountTransaction{} // Unmarshal reads binary data directly into the struct without string parsing if err := proto.Unmarshal(rawBytes, transaction); err != nil { return nil, fmt.Errorf("deserialization failed: %w", err) } if transaction.GetTransactionId() == "" { return nil, fmt.Errorf("missing required field: transaction_id") } return transaction, nil } func main() { // This binary slice is the wire encoding of: // transaction_id: "tx_9988112233", account_status: "ACTIVE", retry_count: 3 // Generated via proto.Marshal on the populated AccountTransaction struct sampleBinaryPayload := []byte{ 10, 13, 116, 120, 95, 57, 57, 56, 56, 49, 49, 50, 50, 51, 51, 18, 6, 65, 67, 84, 73, 86, 69, 24, 3, } start := time.Now() tx, err := processPayload(sampleBinaryPayload) if err != nil { log.Fatalf("processing failure: %v", err) } fmt.Printf("Processed transaction %s in %v\n", tx.GetTransactionId(), time.Since(start)) } The key difference from JSON unmarshaling is in what proto.Unmarshal does not do: it does not tokenize strings, does not map keys via reflection, and does not allocate intermediate string representations. It reads the tag, determines the field type from the compiled schema, and copies raw bytes directly to the target struct field. At high throughput, that distinction in allocation behavior is what drives the difference in GC pressure and tail latency. What This Migration Actually Solves, and What It Does Not Protobuf is not a solution to every distributed systems problem. It will not fix poorly designed service boundaries, reduce round trips caused by chatty interfaces, or compensate for network topology problems. What it specifically addresses is the serialization and deserialization overhead on hot paths where internal services are exchanging high volumes of structured messages. The teams that see the clearest wins are those where profiling has confirmed that serialization CPU time is a meaningful contributor to request latency, and where payload sizes have made bandwidth a real infrastructure cost. If your p99 latency problems trace to database queries, downstream API calls, or lock contention, the Protobuf migration will have minimal impact on those numbers. Start by profiling your highest-traffic internal endpoints. Measure serialization time as a fraction of total request time. Measure payload sizes across a representative sample of production traffic. If the data shows serialization is a genuine bottleneck, the migration is well-justified. If it is not, the operational investment in schema management and tooling upgrades may not pay off on the timeline you need. For the services where it does make sense, the gains are real and durable. Lower CPU utilization, reduced GC pressure, smaller payloads across every internal hop, and strongly typed contracts enforced at build time; these compound over time as traffic grows. Summary The path from JSON to Protobuf is not about chasing a trend. It is a deliberate architectural decision to eliminate serialization overhead on hot internal paths by replacing text parsing with direct binary memory operations. The practical steps are straightforward: audit your highest-traffic internal endpoints, define your .proto schemas with careful attention to field numbering and null semantics, deploy dual-stack services to enable a phased cutover, and establish tooling for schema versioning before your team's first production deployment. The operational costs are real but manageable. Binary streams require schema-aware debugging tools, .proto files need disciplined version management, and the reserved keyword must become part of your deprecation workflow. Teams that treat schema governance as a first-class concern alongside their code absorb these costs smoothly. For distributed systems where internal traffic volume makes serialization overhead measurable, the migration consistently delivers: lower tail latency, reduced bandwidth spend, and contracts that fail loudly at compile time rather than silently at runtime.

In many analytics platforms, there are performance issues that do not always come from complex transformations. Sometimes the bottleneck is much simpler: the same large datasets are being read repeatedly from remote storage. This pattern is common in shared analytics environments. A data engineering job reads a curated dataset to build aggregates. A BI refresh reads the same table again. A data science notebook filters the same records during exploration. Another scheduled workflow joins against the same reference data several times during the day. Each workload may be valid on its own, but together they create repeated remote reads. Over time, this can increase query latency, consume unnecessary infrastructure resources, and make interactive analytics feel slower than expected. Databricks disk cache is designed to help with this type of workload. It stores copies of remote Parquet data files on the local storage of worker nodes so that repeated reads can be served locally instead of fetching the same files again from cloud object storage. This article walks through a practical use case for using Databricks disk cache to improve repeated analytics workloads. The focus is not simply on enabling a feature, but on understanding when disk cache helps, where it fits in a pipeline, and what tradeoffs teams should consider before relying on it. The Use Case: Repeated Reads From Curated Analytics Tables Consider a common analytics setup. A team maintains a curated dataset that is used by multiple downstream workloads. The table is stored in cloud object storage and accessed through Databricks. It is already cleaned, standardized, and partitioned by date. Several jobs and users access this table throughout the day. The dataset supports different types of work: dashboard refreshesscheduled aggregationsexploratory notebooksfeature preparation jobsad hoc analysisdownstream transformation pipelines. The problem is not that the table is poorly designed. The problem is that the same files are repeatedly scanned from remote storage. In this situation, the first read of the data still needs to fetch files from remote storage. However, after the data is cached locally on worker nodes, repeated reads can avoid some of that remote access. For workloads that repeatedly query overlapping data, this can make a noticeable difference. This use case is especially relevant when teams work with large Parquet or Delta tables where the same filtered slices are accessed multiple times. Where Disk Cache Fits in the Pipeline Disk cache is not a replacement for good data modeling, partitioning, or query optimization. It works best as an acceleration layer for workloads that already read reasonably structured data. A practical architecture may look like this: Data Architecture Pipeline With Cache Layer The important point is that disk cache usually adds the most value after data has already been curated. If raw data is messy, unpartitioned, or constantly changing, caching alone will not solve the deeper performance problem. A better pattern is to first create reliable curated datasets and then use disk cache to improve workloads that repeatedly read those datasets. Why Repeated Reads Become Expensive Cloud object storage is highly scalable, but repeatedly reading the same large files still introduces overhead. A query may need to: locate filesread metadatafetch data over the networkdeserialize columnar datascan partitionsapply filterspass data into downstream transformations When one workflow performs this operation, the cost may be acceptable. When several workloads read the same dataset repeatedly, the overhead becomes more visible. This is especially noticeable in interactive analytics. A user may run one query, adjust a filter, run another query, and continue exploring. If every query repeatedly fetches the same underlying files from remote storage, the user experience can degrade quickly. Disk cache helps by keeping frequently accessed data closer to the compute layer. Disk Cache vs Spark Cache One source of confusion is the difference between Databricks disk cache and Apache Spark cache. Spark cache is usually applied manually to a DataFrame or table. It is useful when a specific intermediate result will be reused within the same job or notebook. However, Spark cache requires the developer to decide what to cache and when to unpersist it. Databricks disk cache behaves differently. It works at the file-read level and stores remote Parquet data files locally on worker nodes. When the same data is read again, Databricks can serve it from local disk instead of fetching it again from remote storage. A simple way to think about the difference is this: Spark Cache Developer-controlledApplied to DataFrames or RDDsUseful for reused intermediate resultsRequires explicit cache management. Databricks Disk Cache Managed by DatabricksApplied to remote Parquet/Delta file readsUseful for repeated reads from storageUses local worker disk. In practice, these two caching approaches solve different problems. Spark cache is useful when the same transformed DataFrame is reused multiple times inside a workload. Disk cache is useful when workloads repeatedly scan the same remote Parquet or Delta files. Using the wrong caching strategy can lead to unnecessary memory pressure, unstable performance, or no real improvement. A Practical Example Without Making It Industry-Specific Assume an organization maintains a large curated events table. The table contains activity records from different systems and is used for reporting, operational analytics, and product usage analysis. Several teams query this dataset daily. One dashboard refresh reads the last 30 days of activity. A transformation job reads the same table to calculate weekly aggregates. Analysts use notebooks to filter the data by region, product, and time period. Another pipeline reads the same table to prepare downstream metrics. Even though the consumers are different, many of them repeatedly access the same recent partitions. Without disk cache, these workloads repeatedly read files from remote storage. With disk cache, frequently accessed Parquet files can be stored locally on workers after the first read, allowing later reads to avoid repeated remote fetches. This is not a dramatic redesign of the pipeline. It is an optimization layer that improves workloads with repeated access patterns. When Disk Cache Helps Disk cache is most useful when workloads repeatedly read the same data files. Good candidates include: frequently queried Delta or Parquet tablesdashboard refreshes that scan the same recent partitionsexploratory notebooks that repeatedly filter the same datasetshared reference tables used across multiple joinsiterative analytics workflowsrepeated batch jobs using overlapping input data. The key pattern is repeated access. If every job reads a completely different dataset, disk cache will have limited benefit. If data is accessed once and never reused, the first read still has to fetch the files from remote storage. Disk cache is most effective when the same data is accessed more than once by workloads running on the same or similar compute resources. When Disk Cache May Not Help Much Caching is not a universal performance solution. Disk cache may provide limited improvement when: workloads read data only oncetables change constantlyqueries scan entirely different partitions each timetransformations are CPU-bound rather than I/O-boundjoins and shuffles dominate execution timeclusters are frequently restartedworker nodes are frequently replaced. This last point matters in elastic environments. If workers are decommissioned, local cache data on those workers is lost. The next workload may need to reread data from remote storage. This does not make disk cache unreliable. It simply means teams should understand its behavior before treating it as a guaranteed performance layer. How To Evaluate Whether Disk Cache Is Helping A common mistake is assuming that caching is helping just because it is enabled. A better approach is to compare workload behavior before and after repeated reads. Useful evaluation questions include: Does the second run complete faster than the first run?Are repeated queries reading overlapping data?Is the workload I/O-bound or shuffle-bound?Are the same partitions being scanned repeatedly?Are clusters stable long enough for cache reuse?Are users querying curated tables or constantly changing raw data? Teams should also compare job execution stages. If most time is spent reading remote files, disk cache can help. If most time is spent in large joins, aggregations, or shuffles, caching file reads may only improve part of the workload. Performance tuning should start with measurement, not assumptions. Designing Pipelines To Benefit From Disk Cache To get value from disk cache, the pipeline should be designed in a way that encourages reusable reads. One practical pattern is to separate raw ingestion from curated analytical datasets. Raw data may be inconsistent, frequently updated, and can be less suitable for repeated consumption, while curated datasets are usually cleaner, more stable, and more likely to be accessed repeatedly. A stronger design looks like this: Designing Pipelines for Disk Cache Optimization This design allows disk cache to work on datasets that are already optimized for downstream use. Partitioning also matters. If tables are partitioned in a way that matches query patterns, repeated workloads are more likely to access the same files, if partitioning is poorly aligned with usage patterns then queries may scan too much unnecessary data which would reduce the benefit of caching. For example, if most users query recent data, organizing the table around time-based access patterns can make repeated reads more efficient. Disk cache should be viewed as part of a broader performance strategy, not as a substitute for table design. Operational Considerations There are a few operational details teams should consider before depending heavily on disk cache. First, disk cache depends on local storage on worker nodes. Choosing worker types with local SSD storage can improve caching effectiveness. Second, cache behavior is tied to the lifecycle of the compute environment. If clusters restart frequently, cached data may not persist long enough to benefit repeated workloads. Third, disk cache works best when workloads have predictable reuse patterns. Highly random access patterns are less likely to benefit. Fourth, teams should monitor whether performance improvements are consistent. If query times vary significantly, the issue may not be remote reads alone. The bottleneck may be skewed partitions, insufficient cluster resources, poor join strategy, or inefficient transformations. Finally, caching should not be used to hide poor pipeline design. If a table is too wide, poorly partitioned, or filled with unnecessary historical data, disk cache may improve repeated reads but will not fix the underlying design problem. Avoiding Common Mistakes A few mistakes appear frequently when teams start relying on caching. The first mistake is caching too early in the pipeline. Raw datasets are often unstable and less useful for repeated analytical access. Caching is more valuable after data has been cleaned, standardized, and organized for consumption. The second mistake is confusing disk cache with Spark cache. Spark cache is useful for reused intermediate DataFrames. Disk cache is better suited for repeated reads of remote Parquet or Delta files. The third mistake is ignoring cluster behavior. If compute resources are short-lived, cache reuse may be limited. The fourth mistake is measuring only one query run. Since disk cache is useful for repeated reads, teams should compare cold-read and warm-read behavior rather than judging performance from a single execution. The fifth mistake is treating disk cache as a substitute for optimization. Good partitioning, file sizing, query filtering, and transformation design still matter. Practical Checklist Before depending on disk cache, teams should ask: Are the same datasets read repeatedly?Are workloads reading Parquet or Delta data?Are the tables curated and reasonably stable?Are query patterns predictable?Are clusters stable enough for cache reuse?Are bottlenecks related to file reads rather than shuffles?Are partitions aligned with common access patterns?Are performance gains measured across repeated runs? If the answer to most of these questions is yes, disk cache is likely worth evaluating. If the answer is no, teams should first investigate table design, query plans, file layout, and transformation logic. Conclusion Databricks disk cache can be a useful optimization for analytics workloads that repeatedly read the same Parquet or Delta data from remote storage. It is especially helpful for curated datasets used by dashboards, notebooks, scheduled jobs, and downstream analytics workflows. However, disk cache should not be treated as a general solution for every performance issue. It works best when data access patterns are repeated, compute resources remain stable, and the underlying tables are already designed reasonably well. The biggest lesson is that caching should be intentional. Teams should understand where repeated reads happen, measure cold-read and warm-read behavior, and combine disk cache with good table design, partitioning, and pipeline structure. When used in the right context, disk cache can reduce repeated remote reads and make analytics workloads more responsive. When used without understanding the workload, it becomes just another configuration setting with unclear impact. Reliable analytics performance comes from knowing which bottleneck is actually being solved.

It’s time to meet another member of the DZone community! Abhishek Sharma is a newer face at DZone, but he has already made a great impact. I caught up with him to learn more about his journey into tech and what keeps him curious both in and outside of work. What first got you interested in technology? "What first drew me to technology was seeing how it could solve real business problems. Early in my career, I realized that technology becomes much more interesting when you understand what is happening behind the system — how a business operates, where people struggle, and how technology can simplify that experience. That curiosity stayed with me as I moved from working with enterprise applications into CRM, customer experience, field service, cloud transformation, and enterprise architecture. Over the years, the technologies have changed significantly, but what continues to interest me is the same question: How can we use technology to make a complex business process work better for the people who actually depend on it?" What’s one tool you couldn’t work without? "I would probably say a good architecture diagram — or even a simple whiteboard. My work often involves bringing together business processes, enterprise applications, integrations, data, AI, and operational teams. When a problem becomes complicated, visualizing it usually makes the conversation much easier. Whether I am discussing CRM, field service, inventory, ERP, AI, or integration architecture, putting the end-to-end flow in front of everyone helps people see dependencies that may otherwise be missed. I have learned that sometimes a well-designed diagram can resolve in twenty minutes what several meetings could not." What’s your favorite way to keep your technical skills current? "For me, the best way to stay current is to combine structured learning with practical application. I continue to pursue certifications and explore new capabilities, particularly around Oracle Cloud, Field Service, AI, agentic AI, automation, and enterprise architecture, but I do not like learning technology only at a theoretical level. I learn much more by asking how an emerging technology would actually work in a real enterprise environment. Writing technical articles also helps because it forces me to organize my thinking and challenge my own assumptions. Judging technology awards, engaging with professional organizations, reading industry research, and learning from other architects and practitioners give me perspectives outside my immediate projects as well. Technology changes too quickly to ever say, “I know enough.” Continuous learning has simply become part of the profession for me." In your free time, what do you like to do? "I enjoy hiking and spending time with my kids, especially playing games with them. What I enjoy most about spending time with my kids is seeing the world through their eyes. They often approach a game or a problem with a completely different perspective, and it’s a great reminder that sometimes the best ideas come from looking at familiar things in unfamiliar ways." Hiking sounds wonderful! Do you have any pictures to share? "I have attached a picture of me clicked during one of the Hiking trails near Cuyahoga Falls in Ohio. Hiking is one of my favourite ways to step away from technology and spend time outdoors with his family. For me, it provides a chance to slow down, recharge, and enjoy time away from the demands of work." Check out more of Abhishek's content here.

Almost every conversation about observability budgets I have been in ultimately arrives at the same conclusion: “we need to reduce our telemetry volume.” That sentence is usually followed by a number. Thirty percent. Half. Whatever the finance spreadsheet needs it to be. Then someone says the thing that makes everyone in the room relax. "Good news: most of it’s noise anyway. We can cut the volume and improve the signal at the same time." It is a comforting idea, because it turns an unpleasant budget cut into an engineering improvement. But it only gets you so far. It is true that some of your telemetry is noise, but it’s much less of it than "most." But it doesn’t follow that you can then simply cut volume and automatically improve signal. There is real noise in your telemetry, and I will get to where it lives. But "reduce volume by thirty percent" is not an instruction to remove noise. It is an instruction to remove bytes, and your noise and your signal are made of the same bytes. The target doesn’t differentiate, so what you end up removing is dictated by whatever is easiest to find. What is easy to find is a category. All INFO logs. All user agent strings. Everything below WARN. Categories are easy because your pipeline already knows them, and that is the whole of their appeal. Whether a category happens to be useful or not is a coincidence. So your telemetry is full of junk, but the problem isn't that there is too much of it. It is that by adopting a volume reduction target, you are not looking at whether the telemetry data you cut has any value. Once you hit the byte target, the exercise is seen as a success. Two Axes, Loosely Coupled When you change your telemetry pipeline, two things move. The first is easy: bytes through the pipeline, or active series if it is metrics, or whichever unit your contract happens to price. One number, on a chart, updated hourly. This is what we call volume. The second is what those bytes enable you to find out. Whether, six weeks from now, you can still answer the question in front of you. This is what we commonly call signal, and everything else is noise. It is measurable, but it is not measured in bytes, and it is probably not on any chart you are currently looking at. The two are related, obviously. Delete everything, and both go to zero. But across the range you actually operate in, they are only loosely coupled, because the bytes in your telemetry are not distributed anything like the value. The smallest fields often do the most work. A tenant identifier is a few dozen bytes, and it tells you whether something is impacting everyone or just one customer. A trace ID is thirty-two hex characters, but without it you are correlating your signals by hand, across three browser tabs. If the resource attributes naming the deployment are missing, good luck telling a bad release from a bad node. On the flipside, fields that take the most space frequently do the least. Meanwhile, the ten-thousandth identical stack trace in an hour is several kilobytes and tells you the same thing the first one did. So a lever that operates on bytes will spend most of its effect in the wrong place, and no exchange rate exists that would let you convert one axis into the other. Drawing them as two axes is a crude picture for that reason. But it is still worth doing, because it separates four moves that a byte count reports as only two. Let me walk through each one. Q1: The Free Lunch, Real But Limited This is the noise I promised at the top, and finding it feels great. Every tutorial on making your observability pipeline better has these prominent examples: Kubernetes liveness and readiness probes logging every few seconds, per pod, forever. A debug logger somebody enabled during an incident last quarter, and nobody turned off. The same records shipped twice because a node agent and an application-level exporter both picked them up. Most of this can go. But be careful even here, because a health check is not the same thing as a worthless record. Probe failures and probe latency are how you find a sick node before your users do. What you want to drop is the successful ones, the ninety-nine percent that only ever confirm that nothing is happening. The filter processor will do it: YAML processors: filter/healthchecks: log_conditions: - 'IsMatch(log.attributes["http.route"], "^/(healthz|readyz)$") and log.attributes["http.response.status_code"] == 200' This assumes http.route has been promoted onto the log record; it is a span attribute by default, so on the trace side the equivalent lives under trace_conditions, with a span. prefix instead of log.. That status code check is the difference between Q1 and Q2. Without it, you have removed probe observability rather than probe noise, and you will find that out the next time readiness starts flapping and nothing in the logs can tell you when it began. With it, volume goes down, and signal is untouched, or arguably goes up, because you are no longer scrolling past successful probe traffic to find a real request. Sounds like a good deal, right? This is the quadrant everybody is imagining when they say "most of it is noise anyway." The same trade is available on the retry storm that repeats one stack trace ten thousand times in an hour. The logdedup processor collapses each ten-second window into one record carrying the count, so the storm stops drowning the query you are running, and you can still see how big it was. Finding the rest of this kind of waste means clustering records by shape and looking at what dominates, which is a different class of tool than a filter, and it is the part most volume-reduction programs skip. The challenge is that this quadrant is finite. In my experience, it is somewhere in the range of 10-20%, depending on how neglected the pipeline has been. If your mandate was 30%, you exhaust Q1 in the first week, and then you keep going, because the mandate does not stop when the free lunch does. Q2: Paying With Data Instead of Money So the free lunch got you 15%, the middle of that range, and the mandate was 30%, so the next 15% has to come out of data that somebody might actually need. Which is a good moment to read the mandate again, because almost nobody means it literally. "We need to reduce our telemetry volume by thirty percent" is very rarely a statement about telemetry. It is a statement about an invoice. Does anybody in that meeting actually want fewer log lines? They want a smaller number at the bottom of a bill. Volume is simply the variable their contract happens to be calculated on. The distinction matters because volume reduction and reducing your bill have different solution spaces. Reducing volume by 30% has one family of answers, and every one of them involves deleting something. Reducing observability spend by 30%, has a different set of options, several, and deleting your data is the one with the worst terms. A logging config goes from INFO to WARN and ships with the next release. Retention drops from thirty days to seven. Traces get sampled at 5%: YAML processors: probabilistic_sampler: sampling_percentage: 5 None of these options is free. Each one of them is defensible in isolation, and what makes them defensible is that they have a big impact. INFO is most of your log volume, seven days covers most incidents, and 5% is a perfectly good sample if all you want is a latency distribution. You end up paying the bill twice, but only one of the payments shows up on the invoice. You are also settling the bill in a second currency: answers you will not have, because you didn’t store the data needed for them. Nobody counts that. Nothing fails and nothing alerts, because a trace that was never recorded does not raise anything. When a customer sends an order ID on Thursday, and the trace behind it was one of the ninety-five per cent, the investigation stalls; somebody says we do not have that, and nobody goes back to look at the config change from earlier in the year that caused it to be dropped. My position is that most of this work should not exist. The engineering is fine! The sampler is correct, the retention change is correct, and both do exactly what they say on the tin. It is just that the whole exercise is effort spent making a bad unit price easier to swallow. It's like an old fridge: defrost it, keep the door shut, put less in it, and yes, your bill really does go down every month. Somebody should still go and look at what a new fridge costs. Q3: The Enrichment Nobody Gets To There is a second way to improve signal-to-noise: instead of removing noise, you add signal. You make the data you are already paying for be more useful. Attaching Kubernetes and cloud metadata with the k8sattributes processor, so a log line knows which namespace, deployment, node, and pod produced it. Parsing an unstructured message body into named, queryable fields with OTTL. Making sure trace context actually propagates across the boundary where it currently drops, so your logs and traces can be correlated instead of merely coexisting. Carrying code.file.path and code.line.number on the records that warrant it, so a log line points at the statement that emitted it instead of leaving you to grep the repository for the format string. YAML processors: k8sattributes: extract: metadata: - k8s.namespace.name - k8s.deployment.name - k8s.pod.name - k8s.node.name transform/parse_access_log: log_statements: - context: log statements: - merge_maps(attributes, ExtractPatterns(body, "^(?P<method>\\w+) (?P<path>\\S+) (?P<status>\\d{3}) (?P<duration_ms>\\d+)$"), "insert") These changes make your telemetry substantially more valuable, but they also increase volume. But most of that is cheaper than you would guess. The Kubernetes metadata are resource attributes, written once per batch in OTLP and shared by every record from the same pod, so at the collector's egress they cost a fraction of a byte per record. The parsing is the real exception: you keep the original body alongside the extracted fields, so the record roughly doubles, and no amount of batching recovers that. A bytes-per-day chart shows you none of that. The enrichment that costs almost nothing and the one that doubles every record show up the same way: the budget line went up. So the work never really gets argued about. Nobody is blocking k8sattributes – it ships enabled in half the Helm charts you might install – and most teams already intend to do all of the above. They just do not do it now, because a volume program has a number in it, and programs with numbers in them end when the number is hit. Q1 gets you fifteen percent, Q2 grinds out the rest, somebody screenshots the graph for the quarterly review, and the work is closed. There is no step after "we reduced it by 30%," because reducing it by 30% was the entire brief. Whether your observability spend is value for money is unanswerable while the telemetry is unusable. You can't defend a bill for data nobody can query, and you can't really attack it either, so the argument settles on price – the only number anybody in the room actually has. Enriched telemetry gets used, and usage is evidence. Most of what produces it is unglamorous work: consistent structure, correlation IDs that survive a hop, log levels that mean the same thing across services. But a team that can name the investigations that resolved faster this quarter, and the correlation that did it, walks into the budget meeting with something to say. Q4: The Change You Were Sure About The framework logs the request. Then the middleware logs it, because the framework's version does not carry the tenant. Then the application logs it a third time with slightly different wording, because by that point nobody trusts the other two. Three records, one event, and no reliable way to say which is authoritative. Every one of those lines was added by somebody trying to improve matters, and each has a different team behind it. That is what Q4 actually is, and why I think of it as the backfire. It is not really the stuff that piles up while nobody is looking; that was the double-shipping back in Q1, where either copy is safe to delete because they are identical. These three records differ from one another, and none of them goes without a conversation. Logging whole request and response bodies for completeness is the same story: you add a great deal of data, and the four fields anybody queries end up inside a blob that nothing has parsed. The same thing happens with a processor from the previous section. Take the k8sattributes block from Q3, change nothing about it, and point it at a different pipeline: YAML service: pipelines: metrics: processors: [k8sattributes] On logs, that was enrichment. On metrics, as soon as the backend treats resource identity as series identity, it is a separate series for every pod – and a fresh set of them after every deploy, because pod names churn. That is how a well-meaning label addition takes out a Prometheus. The config did not change, and neither did the intention behind it. Underneath all three is an assumption that more data is the same thing as more signal, and that if the answer is not in there yet then adding should get you closer. It is the same mistake the volume mandate makes, pointed the other way, and I have watched one team make both inside about two years. The awkward thing is that Q3 and Q4 are not separable at the time, and not only on the chart. From the inside, they are the same act: somebody adds something to a pipeline because they are fairly confident it will help. The engineer putting a pod name on a metric is doing what the engineer putting it on a log did. One of them is right. Review will not catch it either, because the reviewer is working from the same information and the same instinct. You need something that checks whether a question actually got easier to answer. What to Govern Instead Put the four quadrants back together, and the problem shows up in one line. Q1 and Q2 both report as a reduction in volume, so dropping probe traffic and dropping the log lines that explain a failure show up in the quarterly review as the same green arrow. Q3 and Q4 both report as volume up, so the enrichment that made an incident tractable and the label that took out your metrics backend are reported as the same red arrow. A bytes-per-day number cannot separate any of that, but it is the number the entire program is steered by. None of which is an argument against governing telemetry. It grows without limit if nobody is watching, somebody has to own the bill, and a team that has never questioned its telemetry costs is not being principled, is just not looking. The argument is about which variable should be on the dashboard. The goal is to try and measure signal, and it is less work than it sounds. Take the ten questions your team actually asks during an incident. Can I segment this failure by tenant? Can I get from this alert to the trace that caused it? Can I tell which deployment introduced it? Write each one as a literal query, in a file, checked into the repository that holds your collector config, and run them in CI against a replay of real telemetry, once with the proposed change and once without. If any answer moves, the build fails. Not just if it comes back empty: sampling does not empty a result; it quietly changes it. That is the difference between Q3 and Q4 made mechanical. The engineer adding pod name to a metric finds out in the pull request instead of during the next incident. It works in reverse too, which is the part that matters for Q3: adding a question and watching it fail is how you justify an enrichment to somebody whose only other number is bytes per day. And if you would rather start with something off the shelf, the Instrumentation Score is an open specification for grading OTLP against semantic conventions and instrumentation best practice, which is a different cut at the same question. Either way: your observability pipeline is probably the only production system you own with no tests on it, and there is no particular reason for that. Changing the Constraints I want to end somewhere slightly uncomfortable, because I do not think this is really a discipline problem or an education problem. Which quadrants you can operate in is dictated by your observability platform's cost model, not by your engineers. If ingest cost scales linearly with bytes, and retention is tiered so that older data becomes slow or expensive or both, then the economics have already made your architectural decisions. Q3 is priced out of existence. Q2 becomes not just permitted but mandatory, because it is the only lever that moves the number anybody is measured on. Your telemetry strategy is a downstream consequence of a pricing page. Teams under that constraint are not making bad choices. They are making the only choices available, and then rationalizing them as noise reduction, because "we improved our signal-to-noise ratio" is a much better sentence than "we deleted data we may need." The interesting question is what changes when volume stops being the binding constraint. When enriching a log record does not require a budget conversation, the matrix opens up. You can attack Q4 aggressively and invest in Q3, which is the combination that actually improves the ratio. Until then, at minimum, name the quadrant. When somebody proposes a pipeline change, ask which of the four it is. It is a five-second question, and I have not yet seen it fail to change the conversation.

When you’re standing up a web app, developers are trying to build required functionality quickly and at a low cost: budgets are still tight, teams are small, and resources are thin. These teams often turn to open-source tools to add PDF viewing functionality, and these libraries work: they give you the basics with simple integration and zero cost. However, whether you’re a scrappy startup or an established organization building new functionality in a platform with a large existing user base, open-source libraries can become a bit of a monkey’s paw: they’ve granted your wish, but the pain comes later. In the case of document processing functionality, this pain comes in the form of integration hell, as you need to add more document capabilities one after another, and the capabilities and dependencies of all the open-source libraries you’ve integrated start to show some cracks. Maintenance grows, user experience suffers, and developers are gently resting their foreheads on the desk. Basic PDF Library vs. Document SDK: What to Choose? The best PDF library for your app depends on how far your document requirements are going to grow, not on how they start. While a basic library renders a document and handles one or two operations well, a document SDK covers the full range a growing application eventually needs: viewing, annotation, editing, redaction, security, and accessibility, from a single license. The table below breaks down the three tiers teams typically move through as document requirements expand. The Three Tiers of PDF and Document Tooling Document tooling generally falls into three tiers, and knowing which one you are in is the first step toward the right decision. Tier Option Best for Limitation1Basic PDF library (open-source, for example, PDF.js, PDFBox, MuPDF)Simple, single-purpose PDF manipulation: view, merge, splitLimited scale, support, and feature breadth. Your team owns maintenance and vulnerability patching.2Point API / cloud document API (for example, Adobe PDF Services, AWS Textract, Azure Document Intelligence, Google Document AI)One specific task like conversion or OCR, fast to prototypeDocuments leave your environment. Per-page costs compound at scale. Adding a second task means fragmented workflows across vendors.3Full document SDK (for example, Apryse)Embedded, scalable document workflows across web, server, and mobileRequires more upfront integration planning than dropping in a single-purpose library. Basic PDF Library (Open-Source) PDF.js, PDFBox, and MuPDF are free, source-available, and fine for a basic viewer. PDF.js is the default free web viewer, built into Firefox, and wins the zero-cost use case outright. MuPDF is a proven rendering engine with decades of use behind it. The limitation shows up once the requirement grows. PDF.js loses fidelity on complex documents, redaction, signatures, and compliance formats like PDF/A and PDF/UA. MuPDF ships as a C-level API with no viewer UI, annotation layer, or forms support, which raises integration cost for anything beyond rendering. All three are single-purpose by design, so a non-trivial workflow means stitching several libraries together and maintaining the glue code between them, with no vendor accountable when something breaks. For a closer look at the tradeoffs between the two models, check out the article open-source vs. proprietary PDF SDKs. Cloud Document API Adobe PDF Services, AWS Textract, Azure Document Intelligence, and Google Document AI get you to a working prototype fast. You call an endpoint, get a converted file or extracted text back, and the vendor manages the scaling behind it. For low or unpredictable volume, pay-as-you-go pricing can make sense. The tradeoff is what happens once you need more than one capability. Each task — conversion, OCR, extraction — tends to live behind a different vendor endpoint, and every one of those endpoints is a place your documents leave your environment before the workflow finishes. Per-page or per-call pricing compounds at production volume, and none of these four hyperscalers offer an air-gapped or offline option if your compliance posture requires it. Full Document SDK A full-document SDK puts extraction, redaction, conversion, and signing behind one engine, instead of several vendors glued together with different conditional code paths. The Apryse PDF SDK runs inside your own environment, whether that is your VPC, on-premises, or fully air-gapped. Document content does not route through a third party to get processed. While Apryse offers a full suite of document-processing capabilities, different tools are licensed as separate add-ons, so you’re not paying for capabilities such as digital signatures or secure redaction unless you actually need them. For example, Docaposte moved its document conversion pipeline to Apryse and saw conversions run 16 times faster than its prior setup. Apryse also runs production document workflows for Dropbox, with more than 700 million users, and Egnyte, across 17,000 businesses. How to Tell When You've Outgrown a Basic PDF Library For developers, it may be time to recognize that your basic PDF library is no longer enough when one or more of these shows up in your backlog: Rendering breaks or slows down on complex or large files your library was not built to handle.Your team is maintaining two or more separate libraries stitched together for one workflow.The roadmap now asks for annotations, redaction, or e-signatures your current library does not support.A compliance requirement shows up, such as SOC 2, ISO 27001, or a data residency rule your current stack cannot meet.Your product needs to render and edit documents consistently across web and mobile, not just one platform.Engineers are spending sprint time patching an open-source dependency instead of building product features. Any one of these on its own might be manageable, but dealing with more usually means the maintenance cost of the current setup has started to exceed the cost of moving to a document SDK. Best PDF Library for Enterprise Apps: What to Evaluate Enterprise-grade performance isn’t just for large organizations. When it’s time to migrate from free libraries to a document SDK, evaluate these criteria to get an enterprise-grade solution: Performance at scale: How does the solution handle concurrency and large, complex files?Feature breadth across the document lifecycle: Does the solution provide viewing, annotation, editing, redaction, and signing from one vendor instead of a different license for each?Security and compliance posture: Look for true content redaction, which permanently removes underlying text and image content rather than masking it visually, plus other document security features such as encryption. On the vendor side, look for independent certifications like SOC 2 and ISO 27001.Support and SLAs: Does the vendor offer a dedicated point of contact for open issues?Deployment control: Can the SDK run on-premises, in your VPC, or fully air-gapped, or does it require routing documents through a vendor's cloud?Licensing model: Does the vendor license cover the full feature set, instead of a separate product and a separate contract for each platform or capability? PDF SDK vs. API: Avoiding Fragmented Workflows An API service solves one task well, but problems can start when the second task arrives. Conversion from Adobe, OCR from AWS Textract, and extraction from Azure Document Intelligence means your application accumulates a different conditional code path for every provider, plus potentially a whole new data residency questionnaire to answer during procurement processes. Check out the article, A Developer’s Guide to Reducing Dependencies, to learn more about vendor consolidation. Apryse consolidates that surface area into a single solution. Office-to-PDF conversion, full-text search across a searchable PDF, redaction, and signing all come from the same engine and the same license, so adding a capability is a configuration change rather than a new vendor integration. That consolidation also keeps document content within your own infrastructure, rather than routing it through several third parties to complete a workflow. Migrating From a Library To an SDK: What It Actually Costs The concern teams raise most often is the cost of moving later, after the app has grown around the library's limitations. That cost is real, but so is the cost of staying on a basic library past the point it fits: slower rendering, an inconsistent user experience, and engineering time spent on patching instead of product work. Let’s look at a real example: Blue Voice built its first version on an open-source React PDF viewer. As the product scaled across police departments, maintaining that PDF functionality started consuming engineering time that the team wanted to spend on its core product instead. After moving to Apryse, according to CTO and co-founder Amit Patankar, "the product felt more polished, our users immediately noticed the difference, and our team could focus on building Blue Voice instead of maintaining a PDF viewer." For a closer look at what the maintenance side of that decision costs over time, read the article The Hidden Costs of Choosing the Wrong PDF Library. If you are ready to compare specific SDKs against your requirements, the Document SDK Buying Guide walks through how to evaluate and buy one. What’s Next for Your Team? Whether you use an open-source document processing library today, or are still planning your project, you can try all Apryse capabilities in a test environment instantly (without needing to talk to sales) by starting your trial. When it’s time to use Apryse in production, contact sales to get licensing that fits your needs. FAQ What is the best PDF library for an enterprise app? The best PDF library for an enterprise app is usually not a basic library at all. Enterprise apps typically need viewing, editing, redaction, and security together, which points toward a full document SDK, like Apryse, rather than a single-purpose library. When do I need a document SDK instead of a basic library? You need a document SDK once your app requires more than one document capability, needs those capabilities to share state, or needs document content to stay inside your own environment for compliance reasons. Apryse offers viewing, editing, redaction, and security together, along with premise-based deployment options. What is the difference between a PDF SDK and a PDF API? A PDF SDK is embedded directly in your application and runs in your own environment. A PDF API is typically a cloud endpoint you call for a single task, which means documents leave your environment, and multiple tasks mean multiple vendor integrations. Is an open-source PDF library good enough for production? An open-source PDF library works well for simple, single-purpose tasks like viewing or merging. It becomes harder to justify once you need broader features, vendor accountability for security patches, or support beyond a community forum. How much does it cost to migrate from a library to an SDK later? The migration cost depends on how much the application has grown around the library's limitations. Teams that wait until rendering issues, maintenance load, or compliance gaps are already affecting users typically face a larger migration than teams that move earlier.

Running Apache Flink on a mainframe sounds odd at first. A modern stream processing engine on a platform most people call legacy? But take a closer look. It is not only possible. It might be a smart move for some of the largest financial institutions in the world. This post explores why some enterprises want Apache Flink on the mainframe, how it could work, and whether it is a brilliant innovation or a technical detour. Disclaimer: The views and opinions expressed in this blog are strictly my own and do not necessarily reflect the official policy or position of my employer. Mainframes Will Still Matter in 203X! A few months ago, I wrote about integrating Apache Kafka with mainframe systems. The blog covered various real-world examples across industries. The key message: Mainframes are still in use. In many organizations, they are not going away. They remain a central part of IT strategy, especially in banking, insurance, and the public sector. But they are not just legacy systems. Modern mainframes such as the IBM z17 offer the latest Telum II processor and support up to 64 terabytes of system memory. The z17 enables very large in‑memory workloads and faster processing for analytics and real‑time use cases. These systems also integrate on‑chip AI acceleration and optional AI‑focused hardware to support machine learning and real‑time decisions directly where mission‑critical data resides, while running modern Linux environments and container platforms. Some companies are still on the mainframe because they cannot easily migrate. But many others do not want to move away. Instead, they modernize around the mainframe. Apache Kafka and Flink play a key role in this journey. They enable a real-time data foundation that connects core systems with modern applications across environments. In future hybrid cloud strategies, this becomes even more critical. Kafka acts as the central nervous system, delivering the right data and context at the right time between on-prem mainframes and cloud-based AI services, including agentic AI and large language models. An event-driven architecture with hybrid streaming replication ensures business-critical decisions are made on fresh, reliable, and contextual information. Mainframe Migration Has Not Happened Ask any architect or CTO in banking. Mainframe migration has been on the roadmap for over two decades. Full replacement of core systems is still rare. However, it is important to distinguish between migration and offloading. Mainframe migration means shutting down mainframe workloads entirely and moving all applications and data to a new platform. There are many reasons: Risk is too highOrganizational resistance is strongMainframe skills are still needed but hard to findSystems are complex and deeply integratedThese applications run reliably and perform well Mainframe offloading, on the other hand, is much more common. It means moving selected workloads, queries, or processing tasks off the mainframe to more flexible and scalable platforms. This reduces load and cost on the mainframe while enabling innovation elsewhere. I have shared several real-world examples of offloading in action, using Kafka, IBM MQ, and Change Data Capture (CDC) tools like IBM IIDR or Precisely to synchronize and replicate data between mainframe systems and the cloud or distributed infrastructure in real time: Mainframe Offloading and Integration Examples. Because of this, many firms choose mainframe integration and a slow lift-and-shift leveraging the Strangler Fig design pattern over migration. Kafka is already helping. Flink is the next step. Apache Flink Meets the Mainframe: Unlikely Combo, Real Potential At first glance, Apache Flink and the mainframe seem like technologies from two different worlds. But combining them can unlock surprising value. What Is Apache Flink? Apache Flink is the leading open-source stream processing engine. It is designed to process high volumes of data continuously and in real time, rather than in batches. Flink is widely used to support use cases like fraud detection, customer personalization, operational monitoring, and data transformation at scale. Many of the largest tech companies and digital natives rely on Flink to process billions of events per day with low latency and high throughput. It supports both event streaming and batch workloads, but its true strength lies in real-time use cases. Here is an example of continuous stream processing leveraging Apache Flink together with OpenAI for Generative AI in real-time: Flink is built for modern environments. It runs natively on Kubernetes, integrates with Apache Kafka for real-time data ingestion, and is commonly deployed in public cloud, private cloud, or hybrid architectures. This makes it an ideal fit for enterprises looking to build fast, intelligent applications on fresh and contextual data. How to Run Apache Flink on the Mainframe? Yes, Apache Flink can run on the mainframe. In fact, it already does. I have already seen this deployed in a real-world environment. A large global financial institution is preparing to invest massively to expand its use of Apache Flink. Running Flink on IBM LinuxONE is a central part of that strategy. This is NOT a lab experiment, but a production-focused initiative. This bank already uses Kafka and Flink in production. Now they want to move Flink compute workloads onto the mainframe. The reason is simple. They already have unused compute on LinuxONE. Running Flink there is cheaper and easier to scale (for some companies) than scaling out other systems. The architecture is modern. IBM LinuxONE runs OpenShift. IBM LinuxONE is a high-performance, enterprise-grade server built on IBM Z architecture. It is designed to run Linux workloads with extreme reliability, scalability, and security. Unlike traditional mainframes focused on COBOL and legacy apps, LinuxONE is optimized for modern Linux applications. Flink is deployed in containers inside OpenShift's Kubernetes infrastructure, just like in any other cloud or data center. From a technical perspective, you need to build Docker images for the s390x architecture to run Apache Flink on IBM LinuxONE. In addition, components like RocksDB, which is used as a state backend in Flink, must be compiled for s390x to ensure full functionality. Why Put Apache Flink on the IBM Mainframe? This is a very valid question! Nobody would buy a mainframe just to run Flink on it. However, this approach offers several benefits for organizations that already own and operate mainframe infrastructure: Available compute resources on the mainframe.Consume data directly from mainframe sources (such as IBM MQ or other integration interfaces) and process it directly on the mainframe; or consume data from external sources such as a Kafka cluster running on x86 infrastructure, enabling flexible integration across hybrid environments.Lower total cost of ownership (TCO) regarding hardware and license costs compared to adding new external x86 servers and bi-directional integration pipelines.Simplified operations within a single, familiar environment.Benefit from IBM actively promoting LinuxONE and driving more workloads onto the platform. Mainframes are not only still alive. They are growing. IBM’s infrastructure business, which includes the mainframe, is doing very well. In Q3 2025, IBM reported 3.6 billion dollars in revenue for the infrastructure segment. That is 17 percent growth. IBM Z alone grew 61 percent. In Q2 2025, infrastructure revenue was 4.14 billion dollars, beating expectations by a wide margin. This is not legacy tech in decline. It is a platform in transformation. A New Chapter for Stream Processing and Mainframes Apache Flink running on the mainframe may sound unusual at first, but it reflects a broader shift in how enterprises think about modernization. The mainframe is not just a legacy system to replace. IBM Mainframe can fit hybrid cloud strategies, especially in highly regulated industries like banking and insurance. Apache Flink brings real-time intelligence. The mainframe brings performance, reliability, and unmatched security. Together, they offer a powerful combination for building fast, contextual, and mission-critical applications, without abandoning existing infrastructure. With Kafka as the backbone and Flink as the engine for real-time processing, organizations can connect mainframe systems with cloud innovation, including advanced AI workloads. This is not just about preserving the past. It is about extending and reusing trusted systems to meet the demands of the future. Enterprises that embrace this model can reduce risk, increase agility, and unlock new value from the heart of their operations.

By Kai Wähner CORE

If you ask Java developers about the concept of ‘Sneaky Throws,’ I am almost sure there will be a couple of opinions that are quite differently expressed, but similar in their meaning. Some will sum it up as being able to throw checked exceptions without declaring them explicitly; others will amend that it means writing functional-style code (lambdas) and being allowed to call methods that throw checked exceptions. Most probably, it will be surely mentioned that there’s a Lombok annotation called exactly @SneakyThrows that solves the problem immediately when put on a method. Last but not least, to outline it in a more pragmatic manner, the concept allows tricking the Java compiler into treating checked exceptions as runtime exceptions. All of these are valid points of view, and to clarify the concept, this article aims to provide a straightforward yet useful approach to handling methods that throw checked exceptions. Let’s jump right in and imagine the following situation. The team is requested to enhance the currently delivered application and implement new functionalities. This obviously happens on a ‘sprint-ly’ basis. Nevertheless, the project has been successfully developed for quite a while now; it also deals with legacy code, and moreover, developers are interacting with other parts of code that were written, let’s say, in a less fortunate manner. Such an example is the class below. Java public class TwoDigitsInteger { private final Integer value; public TwoDigitsInteger(Integer value) { this.value = value; } public boolean isValid() throws NotSetException { if (value == null) { throw new NotSetException("Number value not set."); } return value >= 10 && value <= 99; } public Integer getValue() throws NotSetException { if (value == null) { throw new NotSetException("Number value not set."); } return value; } } Just as its name suggests, it models a two-digit integer number. Instances of this class are immutable; the value is set upon construction, and it declares two methods, one for reading the value — getValue() — and another one for validating it — isValid(). We’re not going to further elaborate on the quality of the code, as it helps in the experiment done. The main issue here, the plot of this article, is the fact that both methods declare a NotSetException as they might throw it under certain circumstances, and even that might be fine unless this Exception hadn’t been a checked one. Java public class NotSetException extends Exception { public NotSetException(String message) { super(message); } } One option (and definitely the one worth taking into account) is to profit and consider the moment a good opportunity to refactor this ‘legacy’ code and at least make the Exception a runtime one. A few unit tests can be written (in case these are missing), then the implementation improved, and focus can be moved on the newly requested features. Nevertheless, for the sake of the experiment in this article, it’s assumed the TwoDigitsInteger class is kept as it currently is and the Exception remains checked. Exception Function Let’s consider a very simple scenario: there is a collection of TwoDigitsIntegers and the intent is to create a string expression that outlines the sum of the numbers. Java List<TwoDigitsInteger> numbers = List.of(new TwoDigitsInteger(10), new TwoDigitsInteger(25), new TwoDigitsInteger(37)); If writing the code as in the test below, Java @Test void sumExpression() { String result = numbers.stream() .map(TwoDigitsInteger::getValue) .map(String::valueOf) .collect(Collectors.joining("+")); Assertions.assertEquals("10+25+37", result); } the Java compiler will complain, saying — Unhandled exception: com.hcd.utilities.NotSetException – as the getValue() method declares a checked Exception and obviously it cannot be used inside a stream. To solve the issue, a try-catch is needed, which makes the code quite difficult to read (and ugly). Not to mention that we’re modifying the state of the joiner as we loop the collection. Java @Test void sumExpression1() { StringJoiner joiner = new StringJoiner("+"); for (TwoDigitsInteger number : numbers) { try { joiner.add(String.valueOf(number.getValue())); } catch (NotSetException e) { throw new RuntimeException(e); } } String result = joiner.toString(); Assertions.assertEquals("10+25+37", result); } In order to overcome this and allow having a fluent API even in situations where checked Exceptions are present, the following ExceptionFunction interface is created. Java @FunctionalInterface public interface ExceptionFunction<T, R, E extends Exception> { R apply(T t) throws E; } It is general enough; it represents a function that accepts one argument (of type T), produces a result (of type R) and when applied, an Exception subclass (of type E) might be thrown. Implementers shall define a single method, which effectively applies the function. Additionally, the following class is defined. Java public final class ExceptionWrapper { public static <T, R, E extends Exception> Function<T, R> apply(ExceptionFunction<T, R, E> function) { return t -> { try { return function.apply(t); } catch (Exception e) { throw new RuntimeException(e); } }; } ExceptionWrapper() { throw new UnsupportedOperationException("No need to be called."); } } When the ExceptionWrapper#apply() method is called, in case an Exception is thrown, it is wrapped into a RuntimeException one and thrown further irrespective of the type of the initial one (the checked Exception case is obviously covered as well, so we’re good). The ExceptionFunction passed as a parameter represents the initial call that is wrapped to overcome the problem. The previously discussed test is modified to use the ExceptionWrapper#apply() method. Not only does it now compile and run successfully, but the code readability is definitely improved. Java @Test void sumExpression() { String result = numbers.stream() .map(ExceptionWrapper.apply(TwoDigitsInteger::getValue)) .map(String::valueOf) .collect(Collectors.joining("+")); Assertions.assertEquals("10+25+37", result); } Exception Predicate Let’s now consider another straightforward scenario, one in which we want to count only the valid two-digit integers that are found in a designated range. Also, for the sake of this experiment, it’s assumed the previous TwoDigitsInteger class is used. As in the previous case, the following piece of code that would do the job doesn’t compile because of the same reason – Unhandled exception: com.hcd.utilities.NotSetException — as the isValid() method declares a checked exception, and it cannot be used inside a stream. Java long count = IntStream.range(0, 150) .mapToObj(TwoDigitsInteger::new) .filter(TwoDigitsInteger::isValid) .count(); Again, assuming the TwoDigitsInteger is needed, one would have to loop through the numbers, check them in a try-catch for checked NotSetExceptions as isValid() declares it, then pack the Exception as a RuntimeException one and throw it further, finally count the valid number. This is already way too complicated even when only enumerating the steps in natural language. To be able to keep the API fluid and use streams when performing checks that declare checked Exception, the next interface is declared. Java @FunctionalInterface public interface ExceptionPredicate<T, E extends Exception> { boolean test(T t) throws E; } It represents a predicate (a boolean-valued function) of one argument that might throw an Exception subclass. The method evaluates the predicate on the given argument and returns true if the input argument matches, or false otherwise. In addition, the following method is added to the ExceptionWrapper class, very similar to the apply() one. Java public static <T, E extends Exception> Predicate<T> test(ExceptionPredicate<T, E> predicate) { return t -> { try { return predicate.test(t); } catch (Exception e) { throw new RuntimeException(e); } }; } When called, it effectively applies the provided predicate. In case an Exception is thrown, it is wrapped into a RuntimeException one and thrown further. The initial code can now be rewritten as below and successfully compiled and executed. Java @Test void count() { long count = IntStream.range(0, 150) .mapToObj(TwoDigitsInteger::new) .filter(ExceptionWrapper.test(TwoDigitsInteger::isValid)) .count(); Assertions.assertEquals(90, count); } Takeaways Although simple and to-the-point, the presented solution comes in very handy, especially when dealing with functions that declare checked Exceptions and are further used in the code that we produce. For sure, other ready-to-use alternatives already exist, an example being the Lombok @SneakyThrows annotation. Personally, I have very rarely included the Lombok library in any of my projects and as Java introduced the records, this becomes even more unlikely to happen in the future. That being said, the structures described in this article are very helpful, lightweight, and easy to understand and use when needed. ExceptionWrapper, ExceptionFunction and ExceptionPredicate source code is part of the asentinel-orm open-source project. To use it, one may either declare the Maven dependency in their pom.xml file (version 1.72.2 is the latest at the moment of this writing) XML <dependency> <groupId>com.asentinel.common</groupId> <artifactId>asentinel-common</artifactId> <version>1.72.2</version> </dependency> or use it directly if considering there’s too much overhead to include the whole library. Resources [1] – asentinel-orm open-source ORM project is here [2] – the picture was taken at ‘Harry Potter Warner Bros. Studios’, near London

By Horatiu Dan CORE