Prometheus: Percentage output with labels from source

I have been trying to work this out for myself and managed to get somewhere.
I have a golang application which extracts workitems from Azure DevOps and creates prometheus metrics.

The Problem

I have a metric called azdo_worktem_tag with labels of project and tag, I wanted to get a percentage of tags prefixed with the value of each unique value of “Team:” over all the tags prefixed with “Team:”.

azdo_workitem_tag{project="xyz" tag="Team: a"} 1
item_metric{project="xyz" tag="Team: b"} 1

In some past solutions I have been met with a lot of errors, or I have been able get something, but the output is labeled as “Value” which made it hard to fix the legends.

The Journey…

I started off with the initial sum by tag:

sum(azdo_workitem_tag{project="xyz",tag=~"Team:.*"}) by (tag)

In all the sites I found discussing this topic they were always mentioning about on and ignoring and group_left _right and a lot of concepts involving those, but I kept getting either a 0 result or an error.

In the end I came upon a stackoverflow answer that helped make sense.

The problem I was not getting my head around was the labels on each metric were crossing over each other and as mentioned in the answer you have to ignore the ones you are filtering.

sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) by (tag) / 
ignoring(project,tag) 
sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"})

The next error I got then made sense “multiple matches for labels: many-to-one matching must be explicit (group_left/group_right)”.

I have to be honest, I lucked out a bit on the next iteration, when I accidentally fired the query too soon! I set group_left and was about to hit shift + 9 but instead hit shift + return and ended up with a desired output.

The Solution

sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) by (tag) / 
ignoring(project,tag) group_left 
sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"})

The output of this provides a percental value (which you then multiply bt 100) with a label of the tag.

By then adding project to the sum by, the metric returned was then also labeled with the project label.

sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) by (tag) / 
ignoring(project,tag) group_left 
sum(azdo_workitem_tag{project="DDC",tag=~"Team:.*"}) 
* 100
visual from grafana explorer

On Reflection

This was a useful learning experience I have a few other situations where I need to use 2 metrics to help me work out some other facts and I think this will help as a good baseline.