Is Gemini 3.8 Flash a new workhorse? Part 2.
In Part 1, we discovered that Gemini 3.8 Flash has a large spread when it comes to response time. In this part, I want to share my investigation of it further.
First of all, let’s see if this delay is time-dependent. In the next section, we are running Gemini 3.8 Flash with Low effort on three prompt sizes.
Time to first token by hour:

Total response time by hour:

Based on those results, we can see that USA morning hours have the biggest spread. But unfortunately, no particular time is protected against unexpected “freezes”.
Considering that “freezes” are more prevalent during the USA morning hours, one can assume that freezes are correlated with higher load on infrastructure serving Gemini 3.8 Flash, since this model is currently the default model for Gemini Workspace and personal Gemini.
Priority inference
To battle the problem above, GCP has introduced a concept of Priority inference:
- Highest criticality: Priority requests get the highest criticality, leading to higher reliability, even during peak load.
- Graceful downgrade: If your traffic exceeds your Priority limits, overflow requests are automatically served at the Standard tier instead of failing. This keeps your application online and helps to ensure business continuity.
- Transparent response: The API response indicates which tier served your request, giving you full visibility into your performance and billing.
Let’s see if switching to Priority inference will help with response time. During our test, we need to monitor to ensure we do not exceed Priority limits.
For this test, we are running the same prompts at the same time (making parallel calls) on Gemini 3.8 Flash on Standard inference and Priority inference using Low effort.
Time to first token: Standard vs Priority

Total response time: Standard vs Priority

As expected, we see that, on average, Priority inference gives faster first token and total response time.
Priority also, in general, has a smaller spread, but surprisingly, we got a large spread on long prompts. Of course, it does not mean that Priority inference has some issue with long prompts — it just shows that while, on average, Priority is faster, it is not “shielded” from “freezing,” and that is why results are sensitive to that.
Provisional throughput
Let’s see if we can take it even further with Provisional throughput.
Provisioned Throughput is a fixed-cost, fixed-term subscription available in several term lengths that reserves throughput for supported generative AI models on the Gemini Enterprise Agent Platform.
Consider using Provisioned Throughput if:
- You are building real-time generative AI production applications, such as chatbots and agents.
- Your critical workloads consistently require high throughput. Throughput measurement depends on the model.
- You want to provide a consistent and predictable experience for users of your applications.
- You want deterministic generative AI costs by paying a fixed monthly or weekly price with control of overages.
Note: You can see all inference options here.
For this test, we are running the same prompts at the same time (making parallel calls) on Gemini 3.8 Flash on Standard inference, Priority inference, and Provisioned Throughput using Low effort.
Time to first token (3 consumption models):

Total response time (3 consumption models)

We see that average response time has not moved much with Provisioned Throughput (which probably means that, on average, with Priority inference you actually get to “hardware” almost immediately without a long waiting queue).
We also see, as expected, that Provisioned Throughput has lower spread. That is good (cause that would be one of the reasons to switch and pay more for Provisioned Throughput).
However, unfortunately, we still see some spread and outliers. That is surprising, cause my expectation was that Provisioned Throughput would provide more or less consistent response time. Of course, we are talking about a probabilistic system, and the same prompt can generate a different amount of thinking tokens on separate calls — but it feels like outliers are too big if it was only for that.
Let’s focus on Provisioned Throughput and repeat a test by doing more calls. In the next test, we are doing calls every 2 min for 24 hours with Provisioned Throughput only.
Time to first token (Provisioned Throughput)

Full response time (Provisioned Throughput)

What we can see from that test:
- Interestingly, time to first token on Low does not depend on prompt size. Full response time, as expected, on average grows with the size of the prompt.
- Unfortunately, we still see significant outliers. That means that even using Provisioned Throughput, we are not protected from “freezes”.
Let’s see if those outliers are time-dependent.
Time to first token (Provisioned Throughput by hour)

Total response time (Provisioned Throughput by hour)

Price
Since we are comparing 3 consumption models, let’s also compare their prices:
- Standard — Per-token (standard rate). As of now and until Dec 31st: $0.75 for input and $3.75 for output (on global)
- Priority inference — Price: Per-token (premium rate). As of now and until Dec 31st: $1.35 for input and $6.75 for output (on global). Basically, almost twice as expensive.
- Provisioned Throughput — Commitment-based (available in 1-week, 1-month, 3-month, and 1-year plans).
Comparing prices for Provisioned Throughput is complicated, cause it fully depends on usage.
In an abstract and ideal scenario, where each second we have a prompt with 100 tokens that generates 100 output tokens and no thinking, Estimation Tool says that we need 1 Generative AI Scale Unit.
The cost of 1 GSU would be $1,200.0 per week or $2,700.00 per month.
With this ideal load, we would use 60,480,000 input tokens and 60,480,000 output tokens in a week. That would cost $272.16 at Standard prices and about double that at Priority. So, even in this ideal situation, Provisioned Throughput would cost more than double Priority and 4x Standard.
The difference becomes lower if 1GSU is taken for a month. In this case, Provisioned Throughput is around +25% of the Priority cost. And since pay-per-use prices are on discount now — one can argue that after the discount period, prices can match.
However, we used an ideal case for the GSU model. In a real use case, demand would fluctuate, and you would need more GSU to cover peak load (or let peak load go to Priority) and have some GSU sit idle sometimes.
What is good is that lack of Provisioned Throughput capacity during peak load does not result in request declines — those requests just go automatically to a lower tier. This means that if you have a stable load 24×7 — you can benefit from Provisioned Throughput covering the “stable part” and let peak load “spill” to the lower tier. In this case, you can benefit from more stable response time for only an extra 25% of cost (and if you go with a 3- or 12-month commitment — no cost difference).
Summary
When it comes to real-time use cases (not batch or background processing), Gemini 3.8 Flash offers 3 consumption models:
- Standard — Flexible, pay-per-use option with no upfront commitment.
- Priority inference — Delivers higher reliability through priority processing while maintaining PayGo flexibility.
- Provisioned Throughput — Provides guaranteed throughput for a commitment period.
As expected, going “up the ladder” in those tiers, you can achieve faster average response time (the difference between Priority and Provisioned was not significant) to both first token and full response.
You also get more stable response time (smaller spread), but still not as stable as you may expect even with Provisioned Throughput.
Outliers remain a major issue in all consumption models, and even Provisioned Throughput is not protected from “freezes”. And, unfortunately, Gemini 3.8 Flash is getting significantly more of them compared to Gemini 3.5 Flash.
For all fairness, it has to be mentioned that Gemini 3.8 Flash is a “smarter” model and requires significantly fewer instructions for the same use cases. And, on average, as shown in Part 1, Gemini 3.8 Flash is also faster.
In the end, it is your decision whether you want/need to “change the horse” and switch from 3.5 to 3.8…
But I hope that now you have more data to make this decision, and know that is now just a simple one “newer model is always better”.
Is Gemini 3.8 Flash a new workhorse? Part 2: Priority inference and Provisional throughput. was originally published in Google Developer Experts on Medium, where people are continuing the conversation by highlighting and responding to this story.