Michele Mancioppi

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

Well yes, but not by itself: as of v0.161.0, the intervalprocessor aggregates only cumulative temporality and passes delta through unchanged. This caught me by surprise (and not just once either...).

Instead you need three components in a row: deltatocumulative to convert, intervalprocessor to drop the cumulative data points you do not want to keep, and cumulativetodelta to convert back.

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

(You could of course interpolate if you want to change the timestamp of the data points, but best you don't go there unless you really have to.)

For delta temporality, aggregation is *much* *more* *interesting*. In the OpenTelemetry Collector there is the intervalprocessor, which sounds like the correct tool for the job, right?

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

Summaries do not carry temporality either, since their count and sum are always cumulative. And speaking of summaries: they are an aberration you should never ever use, because their quantiles cannot be merged.

For cumulative temporality, temporal aggregation is a bit like sampling in traces: you throw away the data points you do not want to keep. And since the data points that remain are self-contained, you are pretty much done.

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

Cumulative temporality means that each data point is effectively self-sufficient, as it contains the entire value; in Prometheus, counters are always cumulative. Delta temporality is a client-side optimization that represents only the change since the last export (hence "delta").

Temporality is a property of the metric, not of the individual data point: sums, histograms and exponential histograms have temporality. Gauges do not, because points you cannot combine have no aggregation to be cumulative or delta about.

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

Spatial aggregation is making a metric "narrower" with fewer timeseries active at the same time.

How you perform the aggregation fundamentally depends on two things: what type of data points you want to aggregate, and their *temporality*. In OpenTelemetry there are two temporalities: #cumulative and #delta.

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

#SpatialAggregation is about merging timeseries: usually you want to throw away some attributes (a.k.a. "labels" in Prometheus) on the data points to reduce #cardinality.

The identity of a timeseries is its set of attributes and their values, so removing attributes merges timeseries. That leaves several data points at more or less the same timestamp, which need to get merged into one.

Let's talk about timeseries #aggregation: the fine art of starting with a smorgasbord of data points, and ending up with fewer. 🧵

There are two fundamental types of timeseries aggregation in #OpenTelemetry: temporal and spatial.

#TemporalAggregation is when you reduce the number of data points of a timeseries, increasing the interval between subsequent ones. That is: you make the timeseries less "dense".

Let's talk about #Trace #sampling, how it allows you to harness the power of distributed tracing without breaking the bank, how it adds inaccuracies, and how you fix that. 🧵

View full thread

I wrote an article about this a little while ago that is still pretty relevant, and there is also an accompanying talk: thenewstack.io/distributed-... www.youtube.com/watch?v=dwWB...

What other tricks do you have to improve the accuracy of sampling?

Let's talk about #Trace #sampling, how it allows you to harness the power of distributed tracing without breaking the bank, how it adds inaccuracies, and how you fix that. 🧵

View full thread

Head and tail sampling reduce the accuracy of your data, over or underestimating error rates and latencies of your services. (Btw, creating animations with LLMs is pretty addictive.)

What can you do to reduce inaccuracies? You additionally record rate, errors and durations (RED) metrics for your services on the full data before sampling. With #OpenTelemetry, you can do trace sampling and RED metrics collection inside the same Collector.

Let's talk about #Trace #sampling, how it allows you to harness the power of distributed tracing without breaking the bank, how it adds inaccuracies, and how you fix that. 🧵

There are two commonplace ways of doing sampling: probabilistic head sampling and tail sampling. With probabilistic head sampling, it is like you throw a die for every trace, and keep it only if the die's face is low enough. With tail sampling, you keep *interesting* traces.

Hello BlueSky! On Monday I am starting a long-running series of threads called #Observability #Animated, in which we dive deep into advanced observability and #OpenTelemetry concepts.

For the first few weeks it is going to be almost daily (I have a good amount of content prepared!) then twice a week. Let me know if you have burning questions or curiosities about #O11y or #OpenTelemetry ;-)

If on Twitter it was tweeting, here on Bluesky is it bleating?

I was jolted awake by this thought btw.

12 June

The regular corruption of TTY terminals in Mac OS is the BSOD of our times, but for my beloved Claude.

vimeo.comThe EnshittificatorDigital products and services keep getting worse. In the new report Breaking Free: Pathways to a fair technological future, the Norwegian Consumer Council has delved…

30 Mar

@drruler.bsky.social it just came to me: I do not recall a snark about a game titled “WarHAMMER” having un-hammered the hammer faction. But I must be misremembering.

Michele Mancioppi reposted

Severin Neumann

@svrnm.com

28 Feb

From the Norwegian Consumer Council: „The Enshittificator“ vimeo.com/1168468796?f... — make it shitty 😭

Michele Mancioppi reposted

OpenTelemetry

@opentelemetry.io

6 Jan

New year resolution: Connect with the #OpenTelemetry community IRL ✨ OTel Unplugged EU - Feb 2 in Brussels → Unconference format → You pick the topics → Direct maintainer access → Real roadmap influence 1 month out. Limited spots: buff.ly/aaUyQsx RT if you're attending or thinking about it!

beta.ideas.lego.comMacross VF-1J Valkyrie | LEGO® IdeasThe VF-1 Valkyrie is one of the most iconic anime mecha designs ever created. This model is inspired by the VF-1J Valkyrie piloted by Hikaru Ichijyo from the le…

28 Dec 2025

Peeps, upvote this please. I need it in my life ASAP! beta.ideas.lego.com/product-idea...

Michele Mancioppi reposted

bit.lyAn Observability Veteran on AI's 'Intoxicating' PotentialGenerative AI is transforming software observability, moving beyond static dashboards to conversational troubleshooting and intelligent telemetry analysis.

13 Nov 2025

Last day of #KubeConNA 2025! 🎨 Meet The Artist, the final (but not least!) Agent0 MtG card celebrating the launch of Agent0. In Dash0, The Artist builds and maintains dashboards and alert rules for you. In MtG, it creates and buffs constructs to help you win. Find The Artist at Booth 710!

Michele Mancioppi reposted

The New Stack

@thenewstack.io

11 Nov 2025

Generative AI is transforming software observability, moving beyond static dashboards to conversational troubleshooting and intelligent telemetry analysis. By @michele.dash0.com, thanks to CNCF and @dash0.com

8 Nov 2025

Come and see what we cooked up! #Observability #MagicTheGathering #OpenTelemetry

Dash0@dash0.com

Hello Internet! Ready for KubeCon? 🔥 To celebrate last week’s launch of #Agent0 - the integrated Agentic #AI inside #Dash0 - we made something we’re pretty stoked about. At #KubeConNA 2025, we’re giving away five custom MtG-style cards. Find us at booth 710 - first teaser: The Seeker & The Oracle!

Michele Mancioppi reposted

28 Oct 2025

Sometimes the best ideas start as simple questions. 💭 That’s how #OpenTelemetry for Dummies - Dash0 Special Edition by Ayooluwa Isaiah and @kaspernissen.xyz was born, to make #observability a little more human, a little clearer, and a little more fun. Free download → www.dash0.com/lp/opentelem...

university.platformengineering.orgObservability for Platform EngineeringGain a solid foundation of modern observability in platform engineering, from essential concepts to practical tooling. Learn to simplify telemetry, navigate open-source solutions, and enforce effectiv...

12 Oct 2025

It’s not supposed to?

Queen Olivia III@queenoliviaiii.com

In honour of spooky month, share a 4 word horror story that only someone in your profession would understand. “The tapes wouldn’t start”

Michele Mancioppi reposted

Kasper Borg Nissen

@kaspernissen.xyz

4 Sept 2025

From my years as a #PlatformEngineer I learned #observability is twofold: watching the infra and enabling devs. We nailed the first, but not the second - that inspired this free course I've built together with @michele.dash0.com and PlatformEnginnering.org! Sign up now! @dash0.com

27 Aug 2025

I laughed way too hard at this

David Buchanan@retr0.id

we will model the end user as a frictionless sphere, in a vacuum

github.comUpcoming changes to the Bitnami catalog (effective August 28th, 2025) · Issue #35164 · bitnami/charts📢 Overview As of August 28th, 2025, the Bitnami public catalog will undergo the following changes: Community catalog Disable images generation for Debian-based images and gradually move existing on...

29 July 2025

R.I.P.nami github.com/bitnami/char...

5 July 2025

If you are in Berlin next week and would like to catch up, ping me on DMs :-)

Dash0@dash0.com

The Dash0 crew is getting ready to reunite for #WeAreDevelopers 2025 in Berlin, and excited to show Dash0 in action! 🇩🇪 Don’t miss our CEO Mirko on July 11, on the Tech Leaders Stage, joining a panel about the AI-Ready Stack. It all kicks off next week. Book a live demo: dash0.link/WeAreDevelop...

21 June 2025

@void.comind.network roast me please

16 June 2025

@tresebrothers.bsky.social I am such a fan of your work! Got hooked by Star Traders: Frontiers, and now enjoying Templar Battleforce Elite. If you went back to Star Traders 4x and make planets less of a chore to invade, I am sure I’d enjoy that title too :-)

13 June 2025

Wait where are the Space 4x games for iOS?

13 June 2025

And I mean single player, I want none of that MMO / idle / micro transactions galore bullshit.

Michele Mancioppi reposted

synyx GmbH & Co. KG

@synyx.bsky.social

19 Apr 2025

Wait, what? You missed @michele.dash0.com's Zero to #OpenTelemetry talk at the #jugka last week? 😲 That'd be a shame, as it was quite ... awesome! 💪 But we got you covered! It took a bit, but here's the fully edited version for your edutainment. 🎓📺 Enjoy! 👉 youtu.be/c6_3of2Y9e8 #Karlsruhe

Michele Mancioppi reposted

o11y.news2025-04-14 - o11y news!

15 Apr 2025

🚨 On-call chaos? Alerts blowing up? Dashboards, DMs, and confusion everywhere? We’ve all been there. That’s why we’re excited to launch our new integration with ilert — making incident response faster, cleaner, and way less stressful. 👉 Check out the integration: s.dash0.com/-ilert

Michele Mancioppi reposted

ilert

@ilert.bsky.social

15 Apr 2025

@dash0.com is now integrated with ilert! 🎉 It is an AI-powered, OpenTelemetry-native observability platform that helps devs and SREs explore logs, metrics, and traces in one seamless experience. Learn more: www.ilert.com/integrations... #Observability #OpenTelemetry #IncidentResponse #DevOps #SRE

Michele Mancioppi reposted

Michael Hausenblas

@mhausenblas.info

14 Apr 2025

Check out this week's o11y newsletter with great content from OpenTelemetry tooling to KubeCon recap: o11y.news/2025-04-14/ #observability

Michele Mancioppi reposted

Vercel

@vercel.com

8 Apr 2025

The Vercel Marketplace adds new integrations from Sentry, Checkly, and Dash0. Use the tools you already trust to monitor, measure, and debug your apps. • Integrated billing • SSO access to provider dashboards • Log Drains from supported providers vercel.com/blog/expandi...

vercel.comExpanding observability on Vercel - VercelThe Vercel Marketplace adds new native integrations from Sentry, Checkly, and Dash0. Use the tools you already trust to monitor, measure, and debug your apps with integrated billing, single sign-on, a...

Michele Mancioppi reposted