Substantially improve OTel D&I

[bhs]

This is a companion issue to #370; from the "preface" there:

In #367 an OTel community member brought up the uncomfortable truth that the diversity of participants in OTel SIGs is – what's the word? – "minimal". (I.e., nearly or actually 100% white males)

The OTel governance committee would like to kick off two parallel efforts: one to periodically measure OTel community D&I, and another to actually improve those measurements over time.

This issue is about actually improving our numbers over time.

In the governance committee (GC) discussion this AM, naturally there was a lot of enthusiasm for the goal here, but (clearly) it will take real work; and we would like to move well beyond token gestures that will have little to no actual impact.

Here's a concrete proposal:

• Until June 18, let's generate ideas (via this issue) for projects/programs/efforts that could help to improve OTel diversity and inclusion. This is a "no bad ideas" brainstorm (well... "almost no bad ideas" :)
• From June 18-July 2, we can debate those ideas
• In early July, the GC can help kick things off (and hopefully other efforts can self-start assuming they're in keeping with community guidelines, regulations like GDPR, etc)

PS / final note: We absolutely do not want the D&I burden to fall on the (way-too-small) portion of the OTel community that actually brings some diversity to our ranks! So, "calling all white dudes": let's try to commit real time to actually make a tangible difference here.

[jkowall]

Otel is largely vendor controlled with some big and small ones involved. The large/controlling vendors (with thousands of employees) could try to find more diverse participants internally to get involved or at least be part of the contributing group. I don't want to name names, but some of those vendors have good diversity stats, while others struggle with that as an organization. I think we can simply approach those vendors, I am happy to do so or someone else can take the lead on it.

Second idea is that at these large vendors they have "people" management who specifically work on this issue, we could invite one or more of them to suggest ways to tackle this problem. Once again I am happy to go down this path and drive it forward.

Final idea is to bring this up to the CNCF level and ask for assistance or suggestions as to how they have tried to tackle this problem in other projects. If they haven't done so, maybe this problem is more widespread and not addressed. Probably something this oversight group could easily drive.

Thanks Ben!

[jkowall]

Yet another idea which is managed by the linux foundation and hence loosely coupled to CNCF. https://github.com/chaoss/wg-diversity-inclusion

[bhs]

One idea – perhaps controversial – would be to require vendors who have a seat on the governance committee to also staff diverse maintainers and approvers on the project (... and, to be clear, the alternative would be to forfeit the GC seat).

[trask]

(uncontroversial) don't refer to generic users/contributors/etc using male pronouns, e.g. user sets his own propagators

[trask]

don't use the terms whitelist and blacklist, instead use something more descriptive that doesn't rely on/imply white=good/allowed black=bad/denied, e.g. allowlist and denylist

[tedsuo]

How, as an OSS community, can we do outreach? OTel does not directly hire or take on internships, which are two primary drivers for real inclusion. Are there examples of other OSS projects having success with organizing and outreach?

[yurishkuro]

@tedsuo the Jaeger project has often participated in Outreachy, which provides internships to minorities. However, the nature of internships is such that people finish them and move on to something else, so I don't think we retained any interns as long-term contributors. In my 5yr experience with open source, people tend to contribute the most when it's part of their day jobs, which makes the suggestions above re vendor commitments (#371 (comment), #371 (comment)) the most promising.

[iredelmeier]

So, I believe that there are (at least) two separate things that can and should be addressed:

1. How can OpenTelemetry be more inclusive within the project and community?
2. How can OpenTelemetry be leveraged to help more people succeed outside the project itself, i.e., in their careers?

While neither of the above directly addresses the problem of OpenTelemetry's diversity numbers, they should both help to create a healthier environment in which people other than cis white men can also thrive.

Also, I realize I've been relatively absent from OpenTelemetry land lately (👋). Apologies in advance if any of the suggestions below are steps that have already been taken!

Some ideas:

Enable (read: automate!) friendly, encouraging GitHub interactions

Contributing to open source is supposed to be awesome! What's a lot less awesome? Feeling ignored or being spoken down to.

Consider the following scenario:

1. A new contributor, "Alice", submits their first PR
2. Their PR fails both the CLA and lint checks
3. "Bob", who has "member" next to their username, comes in and tells them they need to sign the CLA
4. Alice signs the CLA
5. The CLA check now passes, but the lint check still fails
6. Bob swings back to let Alice know that the linting is still broken
7. Alice fixes the lint error
8. Standard code review back-and-forth ensues
9. Eventually, the PR gets accepted and merged

You may be thinking to yourself that this sort of lfow is far from non-standard - and you're right! However, there's still a TON of room for (a) this to be a bad experience for Alice and (b) to improve it for future Alices.

Let's assume that our hypothetical Alice is an experienced software engineer and OSS contributor, despite being new to this particular project. GitHub checks have long since become familiar to her; she doesn't need Bob to let her know about the CLA or lint failures. She was already addressing them in the background, before Bob chimed in! Not only is Bob not really helping, Alice may be starting to wonder why he's assuming that she needs explicit help spoon-fed to her. (Why does Bob think that she doesn't know how PR checks work?)

On the other hand, Bob was likely well-meaning. He's walked a bunch of other new contributors through CLA mayhem, for instance; maybe he even experienced it himself. Additionally, he would have brought up both (CLA and linting) all at once, except he's pretty busy today and noticed them separately. Bob was genuinely trying to help.

Bots can make interactions like this better for everyone.

Let's reimagine that the scenario instead happened as follows:

1. "Alice" submits their first PR
2. While the main CI checks are running, a GitHub Action or similar tool quickly responds to:
   a) Welcome Alice to the community
   b) Say thank you for the PR
3. The CI checks finish, again saying that the the CLA and lint checks have failed
4. For each failure, there's a bot response that:
   • Explains the problem
   • Suggests a common solution (e.g., how to sign the CLA)
   • Tells Alice which contributors to ping for help
5. Alice addresses the CLA and lint issues
6. Bob tells Alice that the PR needs tests
7. Standard code review back-and-forth ensues
8. Eventually, the PR gets accepted and merged

What's changed here? First, the very first response to Alice is friendly! Sure it might be a bot, but it's a bot that's speaking on behalf of real humans. Second, Alice knows that there's nothing specific to her about the response - it's very clearly an automated response that will respond the exact same way to even the most seasoned contributor. The response is also genuinely helpful, saving Bob a bunch of time!

Prioritize "glue work"

The tl;dr explanation of "glue work" is that it's about the work that (a) is often particularly critical to a project's success and (b) tends to be undervalued, especially when it comes to things like performance reviews. (If you haven't yet, I highly, HIGHLY recommend reading the link and/or watching the talk version.) Of note, research has shown that women are disproportionately likely to pick up these sorts of tasks - even though they're then under-rewarded for it.

A case in point is the bot work I mentioned above. You may (hopefully!) have agreed that it sounds like a good idea. If so, how are you going to help make sure that it both gets done, and that the person or people responsible are rewarded accordingly? "Accordingly" in this case means at least as well as they would have been for, say, landing a particularly large feature. Glue work is essential and hard; treat it that way.

Cultivate opportunities that help people grow their careers

Consider this statement from Dare Obasanjo the other day about how what a lot of POC in tech need most these days is career opportunities.

@yurishkuro's comment above about how programs like Outreachy, however awesome they may be, don't especially lead to long-term contributions is very relevant here! Similarly, I've had conversations with other OpenTelemetry people about how they/we don't want to mistakenly (let alone deliberately) take advantage of unpaid labour in the project.

So, back to @yurishkuro's point about how contribution is most sustainable when it's part of someone's day-to-day work, is there maybe a way for OpenTelemetry contributions to also benefit contributors who don't work for vendors? My hypothesis is that OpenTelemetry's ecosystem work can be leveraged to help individuals with the sorts of things that, to put it bluntly, might look good on a resume or in a performance review.

OpenTelemetry is unusual among projects (OSS or otherwise) with respect to how much it necessarily integrates with different languages, frameworks, libraries, tools, etc. Yes, the core pieces of OpenTelemetry itself - the SDKs, the collector, and so on - are critical; but it's hard to realize their value without a sufficiently strong ecosystem. Whether you're on the end user or vendor side, you've likely witnessed this firsthand, e.g., in the difficulties of integrating observability (through OpenTelemetry or not) into your company's code or in onboarding new customers.

A lot of "ecosystem work" requires (or should require) relatively little context on OpenTelemetry itself, relative to whatever entity you're trying to create instrumentation for - and knowledge of the latter is something that potential end user contributions often already have! Authoring a new plugin for something your company actually uses is the sort of contribution that can potentially have a very high ROI for someone, relative to the effort and barriers to entry involved.

I'd personally love to see things like:

• More documentation around adding new instrumentation
• A Kubebuilder-esque project (or set of projects) for OpenTelemetry to make creating and maintaining a new plugin easier (e.g., by providing some CI scaffolding out-of-the-box)
• A pipeline and/or set of recipes that helps contributors demonstrate to their employer's powers-that-be the value of their contributions come performance season, along the lines of Mekka Okereke's awesome advice about "difficulty anchors"

I'm happy to expand on this with more detail if there's interest. In particular, I believe that making such an initiative successful and sustainable would require a nontrivial amount of work - but I also believe that, if done well, it could be awesome to both contributors and OpenTelemetry itself.

Make room for at least one dedicated community manager

...and pay them as though they were an experienced software engineer (or more!).

Tie project leadership to "glue work" and/or "sponsorship"

(This is likely either a more or a less controversial iteration of BHS's suggestion above to require vendors to staff diverse maintainers if they want to be on the governance committee.)

Similar to how many companies' engineer levelling at least ostensibly eventually require more senior people to start helping less experienced teammates grow, membership on the governance and/or technical committees could put more emphasis on growing new contributors.

I know that the Kubernetes community has (successfully 🎉) done a fair amount of work along these lines. Paris Pittman and Josh Berkus have a talk (don't know of a video version, sorry) on how mentorship and sponsorship have scaled within Kubernetes; @sarahnovotny can likely speak more on how some of the Kubernetes lessons can be applied to OpenTelemetry.

[parispittman]

Hi all - we formed a new sig at the cncf level for contributor strategy and we can help give y'all some guidance and learn together if you'd like. I'm happy that you are putting this out there for your community to discuss.

There are numerous outreach activities that can help but I do want to echo giving this work the intentional space it needs. While yes, everyone is going to need to do the work, the space is going to help with organizing and delivering on . Example: Kubernetes has a sig for contributor experience where there is no "one" community manager, or nodejs has a few folks that power a community committee, or some projects do go the route of having a dedicated full/part time person/staff to help.

The contributors to the cncf contributor strategy sig can come to you at one of your community meetings, a dedicated meeting, or come to one of ours to discuss further? We meet bi-weekly on Thursday. #sig-contributor-strategy on cncf slack. Feel free to pop by for q&a at any time. I'll cross post this issue to ours as well so we have it on our radar.

[bhs]

For anybody watching this issue, per the initial issue description, we are entering the following phase:

From June 18-July 2, we can debate [the ideas proposed above]

So, consider this message a reminder to start that process.

[bhs]

Related: #402

[bhs]

Last week, the Governance Committee (GC) discussed the idea of requiring vendors with GC seats to also hit some minimum requirements around the diversity of their OTel maintainer+approver staffing.

GC members raised a number of concerns: that GC members are individuals, not representatives of their respective employers; and further that "their employers" can change (e.g., Sergey and Bogdan both changed jobs in the past year); and further that the GC members often have no direct control over OTel staffing decisions made at their respective employers.

So maybe that idea is DOA. But I am still motivated to find things we can do that will have some "teeth" – the fact of the matter is that some very large, publicly-traded companies are willing to devote significant staffing resources to OTel, and anything we can do to incentive more diversity in that population can have a lot of leverage, especially on senior, semi-permanent members of the community (as opposed to outreach programs to mentor much less experienced engineers).

One idea is to express some requirements around vendor marketing rights for OTel and tie those to staffing diversity. E.g., "The OTel website (and/or an 'About OTel' slide for editorial talks) will prominently feature contributing vendors who meet both a minimum general staffing requirement and a minimum diversity staffing requirement."

Thoughts about this idea?

[lizthegrey]

We also should consider joining Linux Foundation's CommunityBridge program, but expressing a preference for financially sponsoring contributors from underestimated backgrounds.