Internal Analytics review guidelines

This page includes introductory material for an Analytics Instrumentation review. For broader advice and general best practices for code reviews, refer to our code review guide.

Review process

We mandate an Analytics Instrumentation review when a merge request (MR) touches or uses internal analytics code. This includes but is not limited to:

In most cases, an Analytics Instrumentation review is automatically added, but it can also be requested manually if the automations miss the relevant change.

Roles and process

The merge request author should

  • Decide whether a Analytics Instrumentation review is needed. You can skip the Analytics Instrumentation review and remove the labels if the changes are not related to the Analytics Instrumentation domain.
  • If an Analytics Instrumentation review is needed and was not assigned automatically, add the labels ~analytics instrumentation and ~analytics instrumentation::review pending.
  • Use reviewer roulette to assign an Analytics Instrumentation reviewer who is not the author.
  • Assign any other reviews as appropriate.
  • ~analytics instrumentation review does not require a maintainer review.

The Analytics Instrumentation reviewer should

  • Perform a first-pass review on the merge request and suggest improvements to the author.
  • Make sure that no deprecated analytics methods are used.
  • If a change to an event is a part of the review:
  • If a change to a metric is a part of the review:
    • Add the ~database label and ask for a database review for metrics that are based on Database.
    • For a metric’s YAML definition:
      • Check the metric’s description.
      • Check the metric’s key_path.
      • Check the product_section, product_stage, and product_group fields. They should correspond to the stages file.
      • Check the file location. Consider the time frame, and if the file should be under ee.
      • Check the tiers.
    • If a metric was changed or removed: Make sure the MR author notified the Customer Success Ops team (@csops-team), Analytics Engineers (@gitlab-data/analytics-engineers), and Product Analysts (@gitlab-data/product-analysts) by @ mentioning those groups in a comment on the issue for the MR and all of these groups have acknowledged the removal.
    • Make sure that the new metric is available in Service Ping payload, by running: Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values).dig(*'key_path'.split('.')) with key_path substituted by the new metric’s key_path.
  • Approve the MR, and relabel the MR with ~"analytics instrumentation::approved".