Paste Check AB Test Report

Author

Affiliation

Megan Neisler

Product Analytics, Wikimedia Foundation

Published

December 4, 2025

Modified

January 5, 2026

Overview

The Wikimedia Foundation’s Editing team is working on a set of improvements for the visual editor to help new volunteers understand and follow some of the policies necessary to make constructive changes to Wikipedia projects.

This work is guided by the Wikimedia Foundation Annual Plan, specifically by the Wiki Experiences 1.1 objective key result: Increase the rate at which editors with ≤100 cumulative edits publish constructive edits on mobile web by 4%, as measured by controlled experiments (by the end of Q2).

In this AB test, we are evaluating the impact of Paste Check. Paste Check is an edit check that will appear when people paste text into an article they are likely not to have written. This check is an effort to increase the likelihood that the new content people are adding to Wikipedia is aligned with the Movement’s commitment to offering information under a free content license. Multiple paste checks can be shown within an editing session while a user is making their edit. You can find more details about this check on the Project Page.

The Paste Check A/B test was deployed on 8 October 2025 to 22 partner wikis identified in T405422.

Methodology

AB Test Design

The team ran an AB test from 8 October 2025 through 28 November 2025 to determine the impact of presenting Paste Check to eligible editing sessions and evaluate the extent to which the feature, in its current form, warrants being deployed to all wikis.

Specifically, we want to test the following hypothesis:

If we prompt new(er) volunteers pasting text from an external site to confirm whether they wrote the content they are attempting to add, then we will see a decrease in the percentage of new content edits new(er) volunteers publish that are reverted on the grounds of WP:COPYVIO (and related policies). This was identified as the experiment’s primary metric.

During this experiment, 50% of users editing a desktop or mobile main namespace page using Visual Editor were randomly assigned to the test group and could be shown Paste Check if their edit met the specified requirements during their edit, and 50% were randomly assigned to the control group who could not be shown Paste Check.

The test included all mobile web and desktop contributors (both registered and unregistered) to the 22 participating wikis that started an edit with Visual Editor (see full list of participating Wikipedias on the this task description). Users remained in the same test group for the duration of the test. We also limited the analysis to edits completed by unregistered users and users with 100 or fewer edits as those are the users that would be shown Paste Check under the default config settings.

Figure1: Paste Check AB Test Bucketing Overview

As shown in Figure 1, not all edits bucketed in the AB test experiment met the requirements for being shown Paste Check. A Paste Check was only shown if the contributor met the specified requirements while typing their edit. Paste Check was shown at about 28% of all published new content edits in the test group (2,117 edits). In this analysis, we compared all edits that were shown Paste Check to edits that were eligible but not shown Paste Check in the control group (based on instrumentation added in this task. This comparison was done to ensure the analysis is focused on the actual effects of the feature.

Evaluation Plan

We used a set of primary and secondary metrics to evaluate the impact of this feature. We also reviewed a set of guardrails to ensure that the Paste Check was not disruptive to the contributor or to the Wikipedias. These metrics are documented in the task.

For each metric, we reviewed the following dimensions: overall by experiment group (test and control), by platform (mobile web or desktop), by user experience and status, and by partner Wikipedia. We also reviewed some indicators such as edit completion rate by the number of checks shown within a single editing session to determine if there was a significant impact at a certain number of checks presented.

Note: For the user experience analysis, we split newer editors into three experience level groups: (1) unregistered, (2) newcomer (registered user making their first edit on Wikipedia), and (3) Junior Contributor (user that has made between 1 and 100 edits).

Please refer to the data collection notebook notebook for more details on the steps to collect the data reviewed in this report.

Summary of Results

New Content Edit Revert Rate

The revert rate trended lower for edits shown Paste Check; however, we are unable to confirm statistical significance at the 95% confidence level.

We observed a -18% relative decrease [10.5% → 8.6%] in the revert rate for edits shown Paste Check compared to edits eligible but not shown Paste Check. However, the data is insufficient to statistically confirm that this effect was not due to random noise.
Our analysis indicates there is an 85.6% probability that edits shown Paste Check are reverted less frequently than eligible edits in the control group. While this falls short of the 95% certainty threshold, it’s a strong indicator that the feature is decreasing reverts.
Revert rates decreased across all user types, with Paste Check’s magnitude of impact increasing with user experience. We see the highest impact for Junior Contributors (a person that has made between 1 and 100 edits). There was a -24% decrease in the revert rate of edits completed by Junior Contributors shown Paste Check compared to a -9% decrease in the revert rate of edits by Newcomers.
Trends differ by platform. For desktop, we observed a -27% decrease [2.6 percentage points; 9.6% → 7%] in the revert rate of edits shown Paste Check. While on mobile web, there was a +20% increase [3.1 percentage points; 15.5% → 18.6%]. We are unable to confirm if either of these effects are statistically significant.

Edit Completion Rate

Paste check did not cause any significant changes in edit completion rate on either desktop or mobile web. Overall, we observed a -1.9% decrease [1.2 percentage points] in the completion rate of edits shown Paste Check.

We also did not see any significant declines in edit completion rate by user type but data indicates Paste Check’s impact on a user’s likelihood of completing an edit may vary based on user’s experience. There was a +2.7% increase in edit completion rate for unregistered users shown Paste Check and -3.4% decrease for Junior Contributors.

Constructive Edit Rate

Aligned with the revert rate results, results show signs that Paste Check increased constructive edits especially on desktop but we are unable to confirm statistical significance at the 95% confidence level based on available data.

Constructive edits increased by +2% for users in the test group shown Paste Check. While trends appear positive, we do not have sufficient data to confirm this effect was not due to random noise.
Our analysis indicates there is an 85.6% probability that edits shown Paste Check are more constructive than eligible edits in the control group. While this falls short of the 95% certainty threshold, it’s a strong indicator that the feature is increasing constructive edits overall.
Trends differ by platform. On desktop, we observed a +2.9% [90.4% → 93%] increase in constructive edit rate for edits shown Paste Check. While on mobile web, there was -3.7% decrease [84.5% → 81.4%]. Both of these results are statistically inconclusive.

Constructive Retention Rate

There were no significant changes in the constructive edit rate for users shown Paste Check. 6% of contributors in both the test and the control group returned 7 to 14 days to make a constructive edit after making an edit where Paste Check was shown and eligible to be shown.

Guardrail summary

Paste Check does not appear to be negatively impacting an editor’s experience or causing disruption on Wikipedia projects based on a review of the identified guardrail metrics, as described in the task description.

We did not observe any significant decreases in edit completion rate or edit revert rate for edits where Paste Check was shown or overall across all edits.
People elected to keep their pasted text in 56% of edits shown in Paste Check. This aligns with the Edit Check dismissal rate observed for Tone Check and slightly lower than rates observed for Reference Check.
We also confirmed that people are not blocked at a higher rate after being shown Paste Check.

Code

# load packages
shhh <- function(expr) suppressPackageStartupMessages(suppressWarnings(suppressMessages(expr)))
shhh({
    library(lubridate)
    library(ggplot2)
    library(dplyr)
    library(gt)
    library(IRdisplay)
    library(tidyr)
    set.seed(5)
    ## for evaluation
    library(relax)
    library(tibble)
    options(digits = 3)
})
#set preferences
options(dplyr.summarise.inform = FALSE)
options(repr.plot.width = 15, repr.plot.height = 10)

# colorblind color friendly pallette:
cbPalette <- c("#999999", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7")

Primary Metric: New Content Edit Revert Rate

Hypothesis: If we prompt new(er) volunteers pasting text from an external site to confirm whether they wrote the content they are attempting to add, then we will see a ≥4% decrease in the percentage of new content edits new(er) volunteers publish that are reverted on the grounds of WP:COPYVIO (and related policies).

Methodology: We reviewed the proportion of all published new content edits where Paste Check was shown at least once in an editing session and were reverted within 48 hours. This was compared to the revert rate of edits in the control group identifed as eligible but not shown Paste Check.

Note: This analysis does not currently evaluate the reason these edits were reverted. We are still waiting on 2025-11 mediawiki history snapshot to obtain revert comment summary to confirm paste checks reverted on the grounds of WP:COPYVIO; however, a previous baseline analysis indicates that a low percentage of revert summaries (0.3%) include language to indicate that the edit was reverted due to a copyright violation. See those results documented in this task. We may need to explore other methods for confirming if edits contained copyright violations including manual review of a sample of edits or reviewing the frequency of suppressed check as these are often copyright violations.

Code

# load paste check save data
paste_check_publish_data <-
  read.csv(
    file = 'data/paste_check_save_data_AB.tsv',
    header = TRUE,
    sep = "\t",
    stringsAsFactors = TRUE
  )

Code

# Cleaning up dataset and renaming fields to clarify meanings

# Set experience level group and factor levels
paste_check_publish_data <- paste_check_publish_data  |>
  mutate(
    experience_level_group = case_when(
     user_edit_count == 0 & user_status == 'registered' ~ 'Newcomer',
     user_edit_count == 0 & user_status == 'unregistered' ~ 'Unregistered',
      user_edit_count > 0 &  user_edit_count <= 100 ~ "Junior Contributor",
      user_edit_count >  100 ~ "Non-Junior Contributor"   
    ),
    experience_level_group = factor(experience_level_group,
         levels = c("Unregistered","Newcomer", "Non-Junior Contributor", "Junior Contributor")
   ))  

#rename test group field to clarify groups
paste_check_publish_data <- paste_check_publish_data  |>
  mutate(test_group = factor(test_group,
         levels = c('2025-09-editcheck-paste-control', '2025-09-editcheck-paste-test'),
         labels = c("control (eligible but not shown paste check)", "test (paste check shown)")))


#rename platform from phone to mobile web to clarify meaning
paste_check_publish_data <- paste_check_publish_data  |>
  mutate(platform = factor(platform,
         levels = c('phone', 'desktop'),
         labels = c("mobile web", "desktop")))


# rename Wiki values to human readable form

wiki_name_lookup <- c(
  "dewiki" = "German Wikipedia",
  "plwiki" = "Polish Wikipedia",
  "idwiki" = "Indonesian Wikipedia",
  "viwiki" = "Vietnamese Wikipedia",
  "twwiki" = "Twi Wikipedia",
  "simplewiki" = "Simple English Wikipedia",
  "itwiki" = "Italian Wikipedia",
  "nlwiki" = "Dutch Wikipedia",
  "cswiki" = "Czech Wikipedia",
  "ukwiki" = "Ukrainian Wikipedia",
  "ruwiki" = "Russian Wikipedia",
  "bnwiki" = "Bengali Wikipedia",
  "hiwiki" = "Hindi Wikipedia",
  "zhwiki" = "Chinese Wikipedia",
  "arwiki" = "Arabic Wikipedia",
  "fawiki" = "Persian Wikipedia",
  "swwiki" = "Swahili Wikipedia",
  "dagwiki" = "Dagbani Wikipedia",
  "fonwiki" = "Fon Wikipedia",
  "cawiki" = "Catalan Wikipedia",
  "euwiki" = "Basque Wikipedia",
  "glwiki" = "Galician Wikipedia"
)

paste_check_publish_data <- paste_check_publish_data %>%
  mutate(
    wiki = recode(wiki, !!!wiki_name_lookup)
  )

Code

#Set fields and factor levels to assess number of checks shown

paste_check_publish_data <- paste_check_publish_data  |>
  mutate(
    multiple_checks_shown = case_when(
         is.na(n_checks_shown) ~ 'no paste checks',
         n_checks_shown == 1 ~ "one paste check",
         n_checks_shown > 1 ~ "multiple paste checks"
    ) ,
     multiple_checks_shown = factor(multiple_checks_shown ,
         levels = c('no paste checks', 'one paste check', 'multiple paste checks')
        ))
         
# note these buckets can be adjusted as needed based on distribution of data
paste_check_publish_data <- paste_check_publish_data  |>
  mutate(
    checks_shown_bucket = case_when(
     is.na(n_checks_shown) ~ '0',
     n_checks_shown == 1   ~ '1', 
     n_checks_shown == 2  ~ '2',
     n_checks_shown > 2 & n_checks_shown <= 5  ~ "3-5",
     n_checks_shown > 5 & n_checks_shown <= 10  ~ "6-10", 
     n_checks_shown > 10 ~ "over 10" 
    ),
    checks_shown_bucket = factor(checks_shown_bucket ,
         levels = c("0","1","2", "3-5", "6-10", "over 10")
   ))

## Overall

Code

paste_check_reverts_overall <- paste_check_publish_data  |>
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to edit shown or eligible to be shown paste check
    group_by(test_group)  |>
    summarise(n_edits = n_distinct(editing_session),
              n_reverts = n_distinct(editing_session[was_reverted == 1])) %>% # reverted within 48 hours
    mutate(revert_rate = paste0(round(n_reverts/n_edits * 100, 1), "%"))

Code

# plot visualization of overall edit revert rates
dodge <- position_dodge(width=0.9)

p <- paste_check_reverts_overall  |>
    ggplot(aes(x= test_group, y = n_reverts/n_edits, fill = test_group)) +
    geom_col(position = 'dodge') +
    scale_y_continuous(labels = scales::percent) +
    geom_text(aes(label = paste(revert_rate, "\n", n_reverts,"reverted edits"), fontface=2), vjust=1.2, size = 10, color = "white") +
    scale_fill_manual(values= c("#999999", "dodgerblue4"), name = "Experiment Group")  +
    labs (y = "Percent of edits reverted ",
           x = "Experiment Group",
          title = "New content edit revert rate",
           caption = "Limited to published new content edits shown or eligible to be shown Paste Check")  +
    theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        axis.text.x = element_text(size = 24),
        axis.title.x = element_text(margin = margin(t = 20, unit = "pt")),
        legend.position= "none",
        axis.line = element_line(colour = "black")) 
      
p

Overall, new content edits shown Paste Check were reverted less frequently. We observed a -18% relative decrease [-1.9 percentage points; 10.5% → 8.6%] in the revert rate for edits shown Paste Check compared to edits eligible but not shown Paste Check.

Note: The rate observed for the control, 10.5%, is lower than baseline revert rates observed for other Edit Checks we’ve evaluated. For example, there’s 29% revert rate for edits detected as having non-netural tone (see T371158#11220470) and a 26% revert rate for edits published without a reference (see final Reference Check AB test report. This indicates edits identified as having pasted text from an external source are less likely to be reverted than other policy violations.

The low revert rates observed in the experiement have resulted in a low sample size of reverts to review for this experiment, which may impact our ability to confirm statistical signficance. We evaluate this in the “Confirming the impact of Paste Check on revert rate” in the section below.

By if multiple Paste Checks were shown

Code

paste_check_revert_bymultiple <- paste_check_publish_data  |>
    filter(is_new_content == 1 & was_paste_check_shown == 1
          & test_group == 'test (paste check shown)' 
         )  |>
    group_by( multiple_checks_shown)  |>
    summarise(n_edits = n_distinct(editing_session),
              n_reverts = n_distinct(editing_session[was_reverted == 1]))  |> # reverted within 48 hours
    mutate(revert_rate = paste0(round(n_reverts/n_edits * 100, 1), "%"))  |>  
    select(-c(2,3)) %>% # removing granular data columns for publication
    gt()   |>
    tab_header(
    title = "New content edit revert rate by if multiple checks were shown"
      )   |>
    opt_stylize(5) %>%
  cols_label(
    multiple_checks_shown = "Were multiple checks shown?",
    # n_edits = "Number of published new content edits",
    # n_reverts = "Number of edits reverted ",
    revert_rate = "Proportion of new content edits that were reverted"
  )  |>
    tab_source_note(
        gt::md('Limited to published new content edits shown or eligible to shown Paste Check')
    )


display_html(as_raw_html(paste_check_revert_bymultiple))

New content edit revert rate by if multiple checks were shown
Were multiple checks shown?	Proportion of new content edits that were reverted
one paste check	9.8%
multiple paste checks	6.1%
Limited to published new content edits shown or eligible to shown Paste Check

Paste Check was shown to the user more than once in a single editing session in about 10% of all published new content edits in the test group.

Editing sessions shown multiple Paste Checks are less likely to be reverted. For editing sessions shown multiple Paste Checks, there was a -37% decrease [3.7 percentage points; 9.8% → 6.1%] in revert rate compared to edits shown a single Paste Check.

By Platform

Code

paste_check_publish_byplatform <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>%
    group_by(platform, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_reverts = n_distinct(editing_session[was_reverted == 1])) %>% #reverted within 48 hours
    mutate(revert_rate = paste0(round(n_reverts/n_edits * 100, 1), "%")) %>%   
    #select(-c(3,4)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "New content edit revert rate by platform"
      )  %>%
  opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    platform = "Platform",
    n_edits = "Number of published new content edits",
   n_reverts = "Number of edits reverted",
    revert_rate = "Proportion of new content edits that were reverted"
  ) %>%
    tab_source_note(
        gt::md('Limited to published new content edits shown or eligible to shown Paste Check')
    )



display_html(as_raw_html(paste_check_publish_byplatform))

New content edit revert rate by platform
Experiment Group	Number of published new content edits	Number of edits reverted	Proportion of new content edits that were reverted
desktop
2025-09-editcheck-paste-control	1715	164	9.6%
2025-09-editcheck-paste-test	1832	129	7%
phone
2025-09-editcheck-paste-control	317	49	15.5%
2025-09-editcheck-paste-test	285	53	18.6%
Limited to published new content edits shown or eligible to shown Paste Check

We see differing trends per platform. For desktop, we’ve observed a -27% decrease [2.6 percentage points; 9.6% → 7%] in revert rate for edits shown Paste Check.

While on mobile web, there was a 20% increase [3.1 percentage points; 15.5% → 18.6%]; however, there a low absolute number of mobile edits shown or eligible to be shown Paste Check (~ 50 reverts for each test group) that have been reverted and this increase is not statistically significant.

By User experience

Code

paste_check_revert_byuserexp <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>%
    group_by(experience_level_group,test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_reverts = n_distinct(editing_session[was_reverted == 1])) %>% #reverted within 48 hours
    mutate(revert_rate = paste0(round(n_reverts/n_edits * 100, 1), "%")) %>%
    select(-c(3,4)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "New content edit revert rate by user experience"
      )  %>%
   opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    experience_level_group  = "User Status",
    #n_edits = "Number of published new content edits",
    #n_reverts = "Number of edits reverted",
    revert_rate = "Proportion of new content edits that were reverted"
  ) %>%
    tab_source_note(
        gt::md('Limited to published new content edits shown or eligible to shown Paste Check')
    )



display_html(as_raw_html(paste_check_revert_byuserexp))

New content edit revert rate by user experience
Experiment Group	Proportion of new content edits that were reverted
Unregistered
control (eligible but not shown paste check)	11.4%
test (paste check shown)	10.2%
Newcomer
control (eligible but not shown paste check)	9.9%
test (paste check shown)	9%
Junior Contributor
control (eligible but not shown paste check)	10.4%
test (paste check shown)	7.9%
Limited to published new content edits shown or eligible to shown Paste Check

Revert rates decreased across all user types, with Paste Check’s magnitude of impact increasing with user experience.

We see the highest impact for Junior Contributors (a person that has made between 1 and 100 edits). There was a -24% decrease in the revert rate of edits completed by Junior Contributors shown Paste Check compared to a -9% decrease in the revert rate of edits by Newcomers.

By partner Wikipedia

Code

paste_check_revert_bywiki <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>%
    group_by(wiki, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_reverts = n_distinct(editing_session[was_reverted == 1])) %>% 
    mutate(revert_rate = paste0(round(n_reverts/n_edits * 100, 1), "%")) %>%  
    filter(n_edits > 120) %>% # limit to wikis with over 100 published edits
    select(-c(3,4)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "New content edit revert rate by partner Wikipedia"
      )  %>%
  opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    wiki  = "Wikipedia",
    #n_edits = "Number of published new content edits",
    #n_reverts = "Number of edits reverted",
    revert_rate = "Proportion of new content edits that were reverted"
  )  %>%
    tab_source_note(
        gt::md('Limited to wikis with > 100 published new content edits')
    )


display_html(as_raw_html(paste_check_revert_bywiki))

New content edit revert rate by partner Wikipedia
Experiment Group	Proportion of new content edits that were reverted
German Wikipedia
control (eligible but not shown paste check)	16.6%
test (paste check shown)	11%
Persian Wikipedia
control (eligible but not shown paste check)	19%
test (paste check shown)	18.1%
Italian Wikipedia
control (eligible but not shown paste check)	11.2%
test (paste check shown)	14.1%
Russian Wikipedia
control (eligible but not shown paste check)	7.4%
test (paste check shown)	6.2%
Limited to wikis with > 100 published new content edits

We did not log sufficient events at many of the smaller and mid-size wikis to confirm per Wikipedia revert rate results. Many Wikipedias had fewer than 20 total paste check eligible edits that were reverted during the reviewed timefame.

For Wikipedias where we logged a sufficient number of events, we observed decreases in revert rate except at Italian Wikipedia.

Confirming the impact of Paste Check on revert rate

We then used a model to correctly infer the impact of offering Paste Check on the liklihood of a new content edit being reverted within 48 hours. This allows us to confirm if the observed changes detailed above are statistically significant (did not occur due to random chance).

Since multiple edits can be made by the same user, we’ll first calculate the revert rate for each user (proportion of all edits saved by a user that are reverted). Then we’ll us the relax package developed by Mikhail Popov (WMF) to determine Chance To Win along with estimates (with credible/confidence intervals) of lift. This is an implementation of Bayesian and Frequentist engines used in Test Kitchen’s automated analytics.

Code

# calculate the proportion for each user

paste_check_reverts_overall_byuser <- paste_check_publish_data |>
    filter(is_new_content == 1 & was_paste_check_shown == 1) |> #limit to edit shown or eligible to be shown paste check
    group_by(test_group, platform, user_id) |>
    summarise(n_edits = n_distinct(editing_session),
              n_reverts = n_distinct(editing_session[was_reverted == 1])) |> # reverted within 48 hours
    mutate(revert_rate = n_reverts/n_edits) # proportion for each user

Code

# rename field names to align with relax package naming convention
paste_check_reverts_overall_byuser <- paste_check_reverts_overall_byuser |>
  mutate(variation = factor(test_group,
         levels = c("control (eligible but not shown paste check)", "test (paste check shown)"),
         labels = c("control", "treatment")))

paste_check_reverts_overall_byuser$outcome = paste_check_reverts_overall_byuser$revert_rate

Code

overall_impact_reverts <- paste_check_reverts_overall_byuser |>
    analyze_relative_lift(metric_type = "proportion") |>
    gt() |>
  tab_header(
    title = md("**Evaluating Paste Check impact on overall revert rate**"),
    subtitle = md("Difference in Metric (Test Group - Control Group)")
  ) |>
  tab_spanner(
    label = md("**Bayesian Analysis**"),
    columns = c(estimate_bayes, chance_to_win, cred_lower, cred_upper)
  ) |>
  tab_spanner(
    label = md("**Frequentist Analysis**"),
    columns = c(estimate_freq, p_value, conf_lower, conf_upper)
  ) |>
  
  # Rename Columns for clarity ---
  cols_label(
    estimate_bayes = md("Point Estimate"),
    chance_to_win = md("Chance to Win"),
    cred_lower = md("95% CI Lower"),
    cred_upper = md("95% CI Upper"),
    estimate_freq = md("Point Estimate"),
    p_value = md("*p*-value"),
    conf_lower = md("95% CI Lower"),
    conf_upper = md("95% CI Upper")
  ) |>
  
  # pply Formatting (Decimals and CI Grouping) ---
  fmt_number(
    columns = everything(),
    decimals = 3 # Use 3 decimals for precision
  ) |>
  
  # Highlight key finding (Inconclusive) ---
  tab_footnote(
    footnote = md("The 95% intervals cross zero, indicating no statistically conclusive difference."),
    locations = cells_column_labels(columns = c(cred_lower, conf_lower))
  ) %>%
  
  #  Style the table ---
  tab_options(
    table.border.top.color = "lightgray",
    column_labels.border.bottom.color = "black",
    column_labels.border.bottom.width = px(2),
    data_row.padding = px(5)
  )

display_html(as_raw_html(overall_impact_reverts))

Evaluating Paste Check impact on overall revert rate
Difference in Metric (Test Group - Control Group)
Bayesian Analysis				Frequentist Analysis
Point Estimate	Chance to Win	95% CI Lower¹	95% CI Upper	Point Estimate	p-value	95% CI Lower¹	95% CI Upper
−0.099	0.144	−0.282	0.084	−0.110	0.264	−0.302	0.083
¹ The 95% intervals cross zero, indicating no statistically conclusive difference.

The estimates summarized in the table above indicate Paste Check is trending favorably (decreasing revert rate); however, the observed effect is not statistically conclusive as indicated by a high p-value (0.381) and CI that crosses 0.

In other words, while we do see a decrease in revert rate for edits shown Paste Check, the effect is too small to confidently distinguish Paste Check’s effect from random noise.

Some other insights:

The point estimates (Bayesian: -0.099, Frequentist: -0.110) suggest the test group performed slightly better.
The Chance To Win value is the probability that the treatment is better than the control. In this case, results indicate that there is 85.6% probability (1-0.144) that that Paste Check is successfully reducing the revert rate. The most likely impact is an absolute reduction of 0.11 percentage points.
The absolute effect size (1.9 percentage point decrease) is too small relative to the total noise and variation in the data introduced by user and Wikipedia random effects.

See the https://wikitech.wikimedia.org/wiki/Test_Kitchen/Automated_analysis_of_experiments#View_experiment_results for more details on how to interpret these results.

Code

# check by platform numbers

platform_impact_reverts <- paste_check_reverts_overall_byuser |>
    group_by(platform) |>
    group_modify(~ analyze_relative_lift(.x, metric_type = "proportion"))|>
    gt() |>
  tab_header(
    title = md("**Evaluating Paste Check impact on revert rate by platform**"),
    subtitle = md("Difference in Metric (Test Group - Control Group)")
  ) |>
  tab_spanner(
    label = md("**Bayesian Analysis**"),
    columns = c(estimate_bayes, chance_to_win, cred_lower, cred_upper)
  ) |>
  tab_spanner(
    label = md("**Frequentist Analysis**"),
    columns = c(estimate_freq, p_value, conf_lower, conf_upper)
  ) |>
  
  # Rename Columns for clarity ---
  cols_label(
    platform = md("Platform"),
    estimate_bayes = md("Point Estimate"),
    chance_to_win = md("Chance to Win"),
    cred_lower = md("95% CI Lower"),
    cred_upper = md("95% CI Upper"),
    estimate_freq = md("Point Estimate"),
    p_value = md("*p*-value"),
    conf_lower = md("95% CI Lower"),
    conf_upper = md("95% CI Upper")
  ) |>
  
  # pply Formatting (Decimals and CI Grouping) ---
  fmt_number(
    columns = everything(),
    decimals = 3 # Use 3 decimals for precision
  ) |>
  
  # Highlight key finding (Inconclusive) ---
  tab_footnote(
    footnote = md("The 95% intervals cross zero, indicating no statistically conclusive difference."),
    locations = cells_column_labels(columns = c(cred_lower, conf_lower))
  ) %>%
  
  #  Style the table ---
  tab_options(
    table.border.top.color = "lightgray",
    column_labels.border.bottom.color = "black",
    column_labels.border.bottom.width = px(2),
    data_row.padding = px(5)
  )

display_html(as_raw_html(platform_impact_reverts))

Evaluating Paste Check impact on revert rate by platform
Difference in Metric (Test Group - Control Group)
Bayesian Analysis				Frequentist Analysis
Point Estimate	Chance to Win	95% CI Lower¹	95% CI Upper	Point Estimate	p-value	95% CI Lower¹	95% CI Upper
mobile web
0.104	0.714	−0.257	0.464	0.166	0.550	−0.847	1.179
desktop
−0.150	0.071	−0.350	0.050	−0.170	0.262	−0.650	0.310
¹ The 95% intervals cross zero, indicating no statistically conclusive difference.

We are also not able to confirm a statistically significant change in revert rate due to Paste Check for either desktop or mobile web.

Results also confirm the differing trends we observed for desktop and mobile web. On mobile web, the availability of Paste check trends toward increasing revert rate while on desktop it is trending towards decreasing revert rate.

Primary Metric: Edit Completion Rate

We also reviewed edit completion rate to determine the impact of paste check on the frequency users complete their edit. This is used to inform if people intuitively interact with the Paste Check experience in ways that are not disruptive to them.

Methodology We reviewed the proportion of edits where Paste Check was shown at least once during the edit session and that were successfully published (event.action = saveSuccess). These edits were compared to the completion rate of edits in the control group that were eligible but not shown Paste Check, as implemented in T402460.

Code

# load data for assessing edit completion rate
paste_check_completion_rates <-
  read.csv(
    file = 'data/paste_check_completion_data.tsv',
    header = TRUE,
    sep = "\t",
    stringsAsFactors = FALSE
  )

Code

# Set experience level group and factor levels
paste_check_completion_rates <- paste_check_completion_rates %>%
  mutate(
    experience_level_group = case_when(
     user_edit_count == 0 & user_status == 'registered' ~ 'Newcomer',
     user_edit_count == 0 & user_status == 'unregistered' ~ 'Unregistered',
      user_edit_count > 0 &  user_edit_count <= 100 ~ "Junior Contributor",
      user_edit_count >  100 ~ "Non-Junior Contributor"   
    ),
    experience_level_group = factor(experience_level_group,
         levels = c("Unregistered","Newcomer", "Non-Junior Contributor", "Junior Contributor")
   ))  

#rename experiment field to clarfiy
paste_check_completion_rates <- paste_check_completion_rates %>%
  mutate(test_group = factor(test_group,
         levels = c('2025-09-editcheck-paste-control', '2025-09-editcheck-paste-test'),
         labels = c("control (eligible but not shown paste check)", "test (paste check shown)")))

#rename platform from phone to mobile web to clarify meaning
paste_check_completion_rates <- paste_check_completion_rates %>%
  mutate(platform = factor(platform,
         levels = c('phone', 'desktop'),
         labels = c("mobile web", "desktop")))



paste_check_completion_rates  <- paste_check_completion_rates  %>%
  mutate(
    wiki = recode(wiki, !!!wiki_name_lookup)
  )

Code

#Set fields and factor levels to assess number of checks shown

paste_check_completion_rates  <- paste_check_completion_rates %>%
  mutate(
    multiple_checks_shown = 
         ifelse(n_checks_shown > 1, "multiple checks shown", "one check shown"),  
     multiple_checks_shown = factor( multiple_checks_shown ,
         levels = c("one check shown", "multiple checks shown")))
         
# note these buckets can be adjusted as needed based on distribution of data
paste_check_completion_rates  <- paste_check_completion_rates %>%
  mutate(
    checks_shown_bucket = case_when(
     is.na(n_checks_shown) ~ '0',
     n_checks_shown == 1  ~ '1', 
     n_checks_shown == 2  ~ '2',
     n_checks_shown > 2 & n_checks_shown <= 5  ~ "3-5",
     n_checks_shown > 5 & n_checks_shown <= 10 ~ "6-10", 
     n_checks_shown > 10 ~ "over 10" 
    ),
    checks_shown_bucket = factor(checks_shown_bucket ,
         levels = c("0","1","2", "3-5", "6-10","over 10")
   ))

Overall

Code

paste_check_completion_rate_overall <- paste_check_completion_rates %>%
    filter(paste_check_shown == 1 ) %>% #limit to sessions where paste check was shown
    group_by(test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_saves = n_distinct(editing_session[saved_edit > 0])) %>%
    mutate(completion_rate = paste0(round(n_saves/n_edits * 100, 1), "%"))

Code

# plot visualization of overall edit completion rates
dodge <- position_dodge(width=0.9)

p <- paste_check_completion_rate_overall  %>%
    ggplot(aes(x= test_group, y = n_saves/n_edits, fill = test_group)) +
    geom_col(position = 'dodge') +
    scale_y_continuous(labels = scales::percent) +
      geom_text(aes(label = paste(completion_rate), fontface=2), vjust=1.2, size = 10, color = "white") +
    scale_fill_manual(values= c("#999999", "dodgerblue4"), name = "Experiment Group")  +
    labs (y = "Percent of edit attempts completed ",
           x = "Experiment Group",
          title = "Paste Check edit completion rate",
           caption = "Limited to edit attempts shown or eligible to be shown at least one Paste Check")  +
    theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        axis.text.x = element_text(size = 24),
        axis.title.x = element_text(margin = margin(t = 20, unit = "pt")),
        legend.position= "none",
        axis.line = element_line(colour = "black")) 

      
p

Paste check did not cause any signficant changes in edit completion rate. We observed a non-statistically significant -1.9% decrease [1.2 percentage points] in the completion rate of edits shown Paste Check.

By if multiple checks were shown

Code

paste_check_completion_rate_bymulti <- paste_check_completion_rates %>%
    filter(paste_check_shown == 1 &
            test_group == 'test (paste check shown)') %>%
    group_by(test_group, multiple_checks_shown) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_saves = n_distinct(editing_session[saved_edit > 0])) %>%
    mutate(completion_rate = paste0(round(n_saves/n_edits * 100, 1), "%")) %>%   
    gt()  %>%
    tab_header(
    title = "Paste Check edit completion rate by if multiple checks were shown"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment group",
    multiple_checks_shown = "Multiple Paste Checks shown",
    n_edits = "Number of edit attempts shown Paste Check",
    n_saves = "Number of published edits",
    completion_rate = "Proportion of edits saved"
  ) %>%
    tab_source_note(
         gt::md('Limited to edit attempts shown or eligible to be shown at least one Paste Check')
    )


display_html(as_raw_html(paste_check_completion_rate_bymulti))

Paste Check edit completion rate by if multiple checks were shown
Multiple Paste Checks shown	Number of edit attempts shown Paste Check	Number of published edits	Proportion of edits saved
test (paste check shown)
one check shown	8389	5236	62.4%
multiple checks shown	3995	2640	66.1%
Limited to edit attempts shown or eligible to be shown at least one Paste Check

The edit completion rate of edits shown multiple Paste Checks is 6% higher than edits shown a single Paste Check. We observed a similar trend for Tone Check as well.

This may be indicative of the type of edit where multiple checks are shown. For example, edits shown multiple checks are likely longer text edits where the user is more motivated to save their edit.

By number of checks shown

Code

paste_check_completion_rate_bynchecks <- paste_check_completion_rates %>%
    filter(paste_check_shown == 1 & test_group == 'test (paste check shown)')  %>% #limit to paste checks shown and test group
    group_by(test_group, checks_shown_bucket) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_saves = n_distinct(editing_session[saved_edit > 0])) %>%
    mutate(completion_rate = paste0(round(n_saves/n_edits * 100, 1), "%")) %>%  
    ungroup()%>%  
    mutate(n_edits = ifelse(n_edits < 50, "<50", n_edits),
           n_saves = ifelse(n_saves < 50, "<50", n_saves))  %>% #sanitizing per data publication guidelines
    group_by(test_group) %>%  
    gt()  %>%
    tab_header(
    title = "Paste Check edit completion rate by the number of checks shown"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    checks_shown_bucket = "Number of Paste Check shown",
    n_edits = "Number of edit attempts shown Paste Check",
    n_saves = "Number of published edits",
    completion_rate = "Proportion of edits saved"
  ) %>%
    tab_source_note(
        gt::md('Limited to edit attempts shown or eligible to be shown at least one Paste Check')
    )


display_html(as_raw_html(paste_check_completion_rate_bynchecks))

Paste Check edit completion rate by the number of checks shown
Number of Paste Check shown	Number of edit attempts shown Paste Check	Number of published edits	Proportion of edits saved
test (paste check shown)
1	8389	5236	62.4%
2	1991	1279	64.2%
3-5	1459	971	66.6%
6-10	404	283	70%
over 10	141	107	75.9%
Limited to edit attempts shown or eligible to be shown at least one Paste Check

We currently don’t see any increase in edit abandonment rate even if a large number (>3) Paste Checks are shown in a single session. In fact, edit completion rate tends to increase; however, there is smaller number of edit attempts that have been shown over 6 Paste Checks (< 600 edit attempts) making these results more susceptible to noise in the data.

By platform

Code

paste_check_completion_rate_byplatform <- paste_check_completion_rates %>%
    filter(paste_check_shown == 1) %>%
    group_by(platform, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_saves = n_distinct(editing_session[saved_edit > 0])) %>%
    mutate(completion_rate = paste0(round(n_saves/n_edits * 100, 1), "%")) %>% 
    #mutate(n_saves = ifelse(n_saves < 50, "<50", n_saves))%>% #sanitizing per data publication guideline
    #select(-c(3,4)) %>% 
    gt()  %>%
    tab_header(
    title = "Paste Check edit completion rate by platform"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    platform = "Platform",
    n_edits = "Number of edit attempts shown Paste Check",
    n_saves = "Number of published edits",
    completion_rate = "Proportion of edits saved"
  ) %>%
    tab_source_note(
        gt::md('Limited to edit attempts shown or eligible to be shown at least one Paste Check')
    )


display_html(as_raw_html(paste_check_completion_rate_byplatform))

Paste Check edit completion rate by platform
Experiment Group	Number of edit attempts shown Paste Check	Number of published edits	Proportion of edits saved
mobile web
control (eligible but not shown paste check)	1967	1126	57.2%
test (paste check shown)	2172	1194	55%
desktop
control (eligible but not shown paste check)	10187	6747	66.2%
test (paste check shown)	10212	6682	65.4%
Limited to edit attempts shown or eligible to be shown at least one Paste Check

We observed slight but not significant decreases for both edits completed on mobile web and desktop. People editing on mobile web were slightly less likely to complete their edit after being shown Paste Check compared to people on dekstop. Edit completion rate decreased by -3.8% on mobile web and -1.2% on desktop.

By user experience

Code

paste_check_completion_rate_byuserstatus <- paste_check_completion_rates %>%
    filter(paste_check_shown == 1) %>%
    group_by(experience_level_group, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_saves = n_distinct(editing_session[saved_edit > 0])) %>%
    mutate(completion_rate = paste0(round(n_saves/n_edits * 100, 1), "%")) %>%   
    #select(-c(3,4)) %>% #data sanitizing for publication
    gt()  %>%
    tab_header(
    title = "Paste check edit completion rate by editor experience"
      )  %>%
 opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    experience_level_group = "Experiment Group",
    n_edits = "Number of edit attempts shown Paste Check",
    n_saves = "Number of published edits",
    completion_rate = "Proportion of edits saved"
  ) %>%
    tab_source_note(
        gt::md('Limited to edit attempts shown or eligible to be shown at least one Paste Check')
    )


display_html(as_raw_html(paste_check_completion_rate_byuserstatus))

Paste check edit completion rate by editor experience
Experiment Group	Number of edit attempts shown Paste Check	Number of published edits	Proportion of edits saved
Unregistered
control (eligible but not shown paste check)	3833	2116	55.2%
test (paste check shown)	4019	2278	56.7%
Newcomer
control (eligible but not shown paste check)	1929	1056	54.7%
test (paste check shown)	2112	1150	54.5%
Junior Contributor
control (eligible but not shown paste check)	6392	4701	73.5%
test (paste check shown)	6253	4448	71.1%
Limited to edit attempts shown or eligible to be shown at least one Paste Check

We did not observe any significant decline in edit completion rate by user type either but data does indicate Paste Check’s impact on a user’s likelihood of completing an edit may vary based on user’s experience.

There was a 2.7% increase in edit completion rate for unregistered users shown Paste Check and 3.4% decrease for Junior Contributors.

By Partner Wikipedia

Code

paste_check_completion_rate_bywiki <- paste_check_completion_rates %>%
    filter(paste_check_shown == 1) %>%
    group_by(wiki, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_saves = n_distinct(editing_session[saved_edit > 0])) %>%
    mutate(completion_rate = paste0(round(n_saves/n_edits * 100, 1), "%")) %>% 
    filter(n_edits > 200) %>%  #limit to wikis with sufficient events
    #select(-c(3,4)) %>% #data sanitizing for publication
    gt()  %>%
    tab_header(
    title = "Paste Check edit completion rate by Wikipedia"
      )  %>%
 opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    wiki = "Wikipedia",
    n_edits = "Number of edit attempts shown Paste Check",
    n_saves = "Number of published edits",
    completion_rate = "Proportion of edits saved"
  ) %>%
    tab_source_note(
        gt::md('Limited to Wikipedias with at least 200 edit attempts during reviewed timeframe')
    )



display_html(as_raw_html(paste_check_completion_rate_bywiki ))

Paste Check edit completion rate by Wikipedia
Experiment Group	Number of edit attempts shown Paste Check	Number of published edits	Proportion of edits saved
Arabic Wikipedia
control (eligible but not shown paste check)	799	244	30.5%
test (paste check shown)	857	273	31.9%
Catalan Wikipedia
control (eligible but not shown paste check)	410	307	74.9%
test (paste check shown)	449	328	73.1%
Chinese Wikipedia
control (eligible but not shown paste check)	536	330	61.6%
test (paste check shown)	613	376	61.3%
Czech Wikipedia
control (eligible but not shown paste check)	416	283	68%
test (paste check shown)	369	231	62.6%
Dutch Wikipedia
control (eligible but not shown paste check)	604	415	68.7%
test (paste check shown)	549	343	62.5%
German Wikipedia
control (eligible but not shown paste check)	2129	1470	69%
test (paste check shown)	2117	1513	71.5%
Indonesian Wikipedia
control (eligible but not shown paste check)	785	572	72.9%
test (paste check shown)	688	427	62.1%
Italian Wikipedia
control (eligible but not shown paste check)	1651	1058	64.1%
test (paste check shown)	1646	1100	66.8%
Persian Wikipedia
control (eligible but not shown paste check)	629	406	64.5%
test (paste check shown)	719	457	63.6%
Polish Wikipedia
control (eligible but not shown paste check)	661	472	71.4%
test (paste check shown)	731	456	62.4%
Russian Wikipedia
control (eligible but not shown paste check)	1829	1160	63.4%
test (paste check shown)	1768	1157	65.4%
Simple English Wikipedia
control (eligible but not shown paste check)	365	232	63.6%
test (paste check shown)	395	232	58.7%
Ukrainian Wikipedia
control (eligible but not shown paste check)	634	465	73.3%
test (paste check shown)	642	438	68.2%
Vietnamese Wikipedia
control (eligible but not shown paste check)	296	207	69.9%
test (paste check shown)	430	295	68.6%
Limited to Wikipedias with at least 200 edit attempts during reviewed timeframe

Results are slightly more variable on a per wiki basis. However, there are no current signs of significant edit abandonment rate for edits shown Paste Check at any of the Wikipedias.

We observed the highest decrease in edit completion rate was at Indonesian Wikipedia (-14.8%). All decreases are below the -20% threshold we identified as a guardrail for this experiment.

Secondary Metric: Constructive Edit Rate

Hypothesis: A larger proportion of new content edits by Newcomers and Junior Contributors will be constructive because they will be shown a prompt to confirm whether they wrote the content they are attempting to add when pasting text from an external site.

Methodology: The proportion of all published edits by users with ≤100 cumulative edits on a mobile web main namespace that are constructive (not reverted with 48 hours). Similar to revert rate, the analysis was limited to new content edits shown or eligible to be shown Paste Check so we can isolate data to edits that would be impacted by this feature.

Note: This metric is also the WE 1.1 Key Result. We will include Paste Check’s impact on this metric as part of our evaluation of the collective impact of interventions deployed under WE 1.1 on this metric.

Overall

Code

paste_check_constructive_overall <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to eligible edits
    group_by(test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_const = n_distinct(editing_session[was_reverted == 0])) %>% #limit to new content edits without a refernece
    mutate(construtive_edit_rate = paste0(round(n_const/n_edits * 100, 1), "%"))

Code

# plot visualization of overall edit completion rates
dodge <- position_dodge(width=0.9)

p <- paste_check_constructive_overall %>%
    ggplot(aes(x= test_group, y = n_const/n_edits, fill = test_group)) +
    geom_col(position = 'dodge') +
    scale_y_continuous(labels = scales::percent) +
    geom_text(aes(label = paste(construtive_edit_rate), fontface=2), vjust=1.2, size = 10, color = "white") +
    scale_fill_manual(values= c("#999999", "dodgerblue4"), name = "Experiment Group")  +
    labs (y = "Percent of edits that were constructive ",
           x = "Experiment Group",
          title = "Overall constructive edit rate",
           caption = "Limited to published new content edits shown or eligible to be shown Paste Check")  +
  theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        axis.text.x = element_text(size = 24),
        axis.title.x = element_text(margin = margin(t = 20, unit = "pt")),
        legend.position= "none",
        axis.line = element_line(colour = "black")) 

      
p

Constructive edits increased by 2% [1.9 percentage points] for users in the test group shown Paste Check

This aligns with the decrease we observed in revert rate, since constructive edit rate is defined as the inverse of revert rate (edits not reverted). An 18% relative decrease in the revert rate results in a 2.1% relative increase in the constructive edit rate.

If we don’t limit to eligible Paste Check edits, we did not observ any changes in constructive edits (80% constructive edit rate in both the test and control groups).

Platform

Code

paste_check_constructive_byplatform <- paste_check_publish_data %>%
    filter(is_new_content == 1 &was_paste_check_shown == 1) %>%
    group_by(platform, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_const = n_distinct(editing_session[was_reverted == 0])) %>% 
    mutate(constructive_edit_rate = paste0(round(n_const/n_edits * 100, 1), "%")) %>%   
    select(-c(3,4)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "Constructive edit rate by platform"
      )  %>%
  opt_stylize(5) %>%
  cols_label(
    test_group = "Test Group",
    platform = "Platform",
    #n_edits = "Number of published new content edits",
   # n_const = "Number of constructive edits",
    constructive_edit_rate = "Proportion of new content edits that were constructive"
  ) %>%
    tab_source_note(
        gt::md('Limited to published new content edits shown or eligible to shown Paste Check')
    )



display_html(as_raw_html(paste_check_constructive_byplatform ))

Constructive edit rate by platform
Test Group	Proportion of new content edits that were constructive
desktop
2025-09-editcheck-paste-control	90.4%
2025-09-editcheck-paste-test	93%
phone
2025-09-editcheck-paste-control	84.5%
2025-09-editcheck-paste-test	81.4%
Limited to published new content edits shown or eligible to shown Paste Check

Also similar to revert rate, we see differing trends on mobile web compared to desktop. Constructive edit rates on desktop increased while they decreased on mobile web.

We observed a -3.7% decrease on mobile web while on desktop we observed a 2.9% increase. Further investigation of edits shown Paste Check on mobile web and reverted might help provide more insight on why this decrease in constructive edits is occuring.

By User Experience

Code

paste_check_constructive_byexp <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>%
    group_by(experience_level_group, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_const = n_distinct(editing_session[was_reverted == 0])) %>% 
    mutate(constructive_edit_rate = paste0(round(n_const/n_edits * 100, 1), "%")) %>%   
    select(-c(3,4)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "Constructive edit rate by user experience"
      )  %>%
  opt_stylize(5) %>%
  cols_label(
    test_group = "Test Group",
    experience_level_group = "User type",
    #n_edits = "Number of published new content edits",
   # n_const = "Number of constructive edits",
    constructive_edit_rate = "Proportion of new content edits that were constructive"
  ) %>%
    tab_source_note(
        gt::md('Limited to published new content edits shown or eligible to shown Paste Check')
    )



display_html(as_raw_html(paste_check_constructive_byexp ))

Constructive edit rate by user experience
Test Group	Proportion of new content edits that were constructive
Unregistered
control (eligible but not shown paste check)	88.6%
test (paste check shown)	89.8%
Newcomer
control (eligible but not shown paste check)	90.1%
test (paste check shown)	91%
Junior Contributor
control (eligible but not shown paste check)	89.6%
test (paste check shown)	92.1%
Limited to published new content edits shown or eligible to shown Paste Check

Constructive edit rate increase for unregistered users and registered Newcomers and Junior Contributors shown Paste Check. We observed the highest impact for Junior Contributors (2.8% increase in constructive edits).

By Partner Wikipedia

Code

paste_check_constructive_bywiki <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>%
    group_by(wiki, test_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_const = n_distinct(editing_session[was_reverted == 0])) %>% 
    mutate(constructive_edit_rate = paste0(round(n_const/n_edits * 100, 1), "%")) %>% 
    filter(n_edits > 100) %>%  #limit to wikis with sufficient events
    select(-c(3,4)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "Constructive edit rate by Wikipedia"
      )  %>%
  opt_stylize(5) %>%
  cols_label(
    test_group = "Test Group",
    wiki = md("**Wikipedia**"),
    #n_edits = "Number of published new content edits",
    #n_const = "Number of constructive edits",
    constructive_edit_rate = "Proportion of new content edits that were constructive"
  )  %>%
    tab_source_note(
        gt::md('Limited to wikis with > 100 published new content edits')
    )


display_html(as_raw_html(paste_check_constructive_bywiki ))

Constructive edit rate by Wikipedia
Test Group	Proportion of new content edits that were constructive
Catalan Wikipedia
control (eligible but not shown paste check)	95.1%
test (paste check shown)	91.1%
German Wikipedia
control (eligible but not shown paste check)	83.4%
test (paste check shown)	89%
Persian Wikipedia
control (eligible but not shown paste check)	81%
test (paste check shown)	81.9%
Indonesian Wikipedia
control (eligible but not shown paste check)	99.1%
test (paste check shown)	99.1%
Italian Wikipedia
control (eligible but not shown paste check)	88.8%
test (paste check shown)	85.9%
Dutch Wikipedia
control (eligible but not shown paste check)	96.4%
Polish Wikipedia
control (eligible but not shown paste check)	83.7%
test (paste check shown)	91.7%
Russian Wikipedia
control (eligible but not shown paste check)	92.6%
test (paste check shown)	93.8%
Ukrainian Wikipedia
control (eligible but not shown paste check)	95.8%
test (paste check shown)	94.8%
Limited to wikis with > 100 published new content edits

For Wikipedias where we logged a sufficient number of events, we observed increases in constructive edit rate except at Italian Wikipedia where there was a -3.3% non-statisticaly significant decrease [ -2.9 percentage points; 88.8% → 85.9%] in constructive edits.

Confirming impact of Past Check on constructive edit rate

We also modeled the impact of Paste Check on constructive edits rates to confirm the magnitude and direction of Paste Check’s effect on a user completing a higher proportion of constructive edits. This helps account for random effects of the user and wiki.

Code

paste_check_constructive_overall_byuser <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to eligible edits
    group_by(test_group, platform, user_id) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_const = n_distinct(editing_session[was_reverted == 0])) %>% #edits not reverted
    mutate(constructive_edit_rate = n_const/n_edits)

Code

# rename test group field names to align with relax package naming convention
paste_check_constructive_overall_byuser  <- paste_check_constructive_overall_byuser |>
  mutate(variation = factor(test_group,
         levels = c("control (eligible but not shown paste check)", "test (paste check shown)"),
         labels = c("control", "treatment")))

Code

# create new column name to align with relax package naming
paste_check_constructive_overall_byuser$outcome = paste_check_constructive_overall_byuser$constructive_edit_rate

Code

# overall impact
overall_impact_const_edits <- paste_check_constructive_overall_byuser |>
    analyze_relative_lift(metric_type = "proportion") |>
    gt() |>
  tab_header(
    title = md("**Evaluating Paste Check impact on overall constructive edit rate**"),
    subtitle = md("Difference in Metric (Test Group - Control Group)")
  ) |>
  tab_spanner(
    label = md("**Bayesian Analysis**"),
    columns = c(estimate_bayes, chance_to_win, cred_lower, cred_upper)
  ) |>
  tab_spanner(
    label = md("**Frequentist Analysis**"),
    columns = c(estimate_freq, p_value, conf_lower, conf_upper)
  ) |>
  
  # Rename Columns for clarity ---
  cols_label(
    estimate_bayes = md("Point Estimate"),
    chance_to_win = md("Chance to Win"),
    cred_lower = md("95% CI Lower"),
    cred_upper = md("95% CI Upper"),
    estimate_freq = md("Point Estimate"),
    p_value = md("*p*-value"),
    conf_lower = md("95% CI Lower"),
    conf_upper = md("95% CI Upper")
  ) |>
  
  # pply Formatting (Decimals and CI Grouping) ---
  fmt_number(
    columns = everything(),
    decimals = 3 # Use 3 decimals for precision
  ) |>
  
  # Highlight key finding (Inconclusive) ---
  tab_footnote(
    footnote = md("The 95% intervals cross zero, indicating no statistically conclusive difference."),
    locations = cells_column_labels(columns = c(cred_lower, conf_lower))
  ) %>%
  
  #  Style the table ---
  tab_options(
    table.border.top.color = "lightgray",
    column_labels.border.bottom.color = "black",
    column_labels.border.bottom.width = px(2),
    data_row.padding = px(5)
  )

display_html(as_raw_html(overall_impact_const_edits))

Evaluating Paste Check impact on overall constructive edit rate
Difference in Metric (Test Group - Control Group)
Bayesian Analysis				Frequentist Analysis
Point Estimate	Chance to Win	95% CI Lower¹	95% CI Upper	Point Estimate	p-value	95% CI Lower¹	95% CI Upper
0.012	0.852	−0.011	0.035	0.012	0.296	−0.011	0.035
¹ The 95% intervals cross zero, indicating no statistically conclusive difference.

Similar to the revert rate analysis, results suggest a positive effect of Paste Check on constructive edits; however, the effect is too small to confirm statistical significance.

Both models show a slight increase in the constructive edit rate when the Paste Check is shown.
The probability that the Test Group truly increased the constructive rate is 85.2%. This is a good indicator that Paste Check has a postive effect on constructive edits overall.
However, both the Bayesian Credible Interval and the Frequentist Confidence Interval contain zero indicating that we cannot confirm statistical signficance.

Code

# check by platform numbers

platform_impact_constr_edits <- paste_check_constructive_overall_byuser |>
    group_by(platform) |>
    group_modify(~ analyze_relative_lift(.x, metric_type = "proportion"))|>
    gt() |>
  tab_header(
    title = md("**Evaluating Paste Check impact on constructive edit rate by platform**"),
    subtitle = md("Difference in Metric (Test Group - Control Group)")
  ) |>
  tab_spanner(
    label = md("**Bayesian Analysis**"),
    columns = c(estimate_bayes, chance_to_win, cred_lower, cred_upper)
  ) |>
  tab_spanner(
    label = md("**Frequentist Analysis**"),
    columns = c(estimate_freq, p_value, conf_lower, conf_upper)
  ) |>
  
  # Rename Columns for clarity ---
  cols_label(
    platform = md("Platform"),
    estimate_bayes = md("Point Estimate"),
    chance_to_win = md("Chance to Win"),
    cred_lower = md("95% CI Lower"),
    cred_upper = md("95% CI Upper"),
    estimate_freq = md("Point Estimate"),
    p_value = md("*p*-value"),
    conf_lower = md("95% CI Lower"),
    conf_upper = md("95% CI Upper")
  ) |>
  
  # pply Formatting (Decimals and CI Grouping) ---
  fmt_number(
    columns = everything(),
    decimals = 3 # Use 3 decimals for precision
  ) |>
  
  # Highlight key finding (Inconclusive) ---
  tab_footnote(
    footnote = md("The 95% intervals cross zero, indicating no statistically conclusive difference."),
    locations = cells_column_labels(columns = c(cred_lower, conf_lower))
  ) %>%
  
  #  Style the table ---
  tab_options(
    table.border.top.color = "lightgray",
    column_labels.border.bottom.color = "black",
    column_labels.border.bottom.width = px(2),
    data_row.padding = px(5)
  )

display_html(as_raw_html(platform_impact_constr_edits))

Evaluating Paste Check impact on constructive edit rate by platform
Difference in Metric (Test Group - Control Group)
Bayesian Analysis				Frequentist Analysis
Point Estimate	Chance to Win	95% CI Lower¹	95% CI Upper	Point Estimate	p-value	95% CI Lower¹	95% CI Upper
mobile web
−0.031	0.219	−0.109	0.047	−0.031	0.517	−0.207	0.144
desktop
0.016	0.919	−0.007	0.040	0.017	0.300	−0.036	0.069
¹ The 95% intervals cross zero, indicating no statistically conclusive difference.

The results show no statistically conclusive change from Paste Check on either platform, with conflicting trends suggesting a different user experience depending on the device.

On mobile web, the availability of Paste Check trends toward decreasing constructive edits while on desktop it is trending towards increasing constructive edits.

Secondary Metric: Distinct users that publish a reverted edit

Hypothesis: Newcomers and Junior Contributors will be more aware of the need to consider whether the text they’re pasting from an external site into a main article namespace is at risk of copyright violations.

Methodology: The proportion of newcomers and Junior Contributors shown or eligible to be shown Paste Check that publish at least one new content edit that was reverted. This metric is similar to the revert rate analysis except that it looks at proportion of distinct editors versus distinct edits. There were no significant differences in the results reported in Primary Metric 1: Revert rate section as the majority of newcomers and Junior Contributors posted just one new content edit during the reviewed time period. See overall results below.

Overall

Code

paste_check_reverts_byuser_overall <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to eligible edits
    group_by(test_group) %>%
    summarise(n_users = n_distinct(user_id),
              n_users_revert = n_distinct(user_id[was_reverted == 1])) %>% #limit to new content edits without a refernece
    mutate(revert_rate = paste0(round(n_users_revert/n_users * 100, 1), "%"))

Code

# plot visualization of overall users reverted
dodge <- position_dodge(width=0.9) 

p <- paste_check_reverts_byuser_overall  |>
    ggplot(aes(x= test_group, y = n_users_revert/n_users, fill = test_group)) +
    geom_col(position = 'dodge') +
    scale_y_continuous(labels = scales::percent) +
    geom_text(aes(label = paste(revert_rate, "\n", n_users_revert,"users reverted"), fontface=2), vjust=1.2, size = 10, color = "white") +
    scale_fill_manual(values= c("#999999", "dodgerblue4"), name = "Experiment Group")  +
    labs (y = "Percent of distinct users reverted ",
           x = "Experiment Group",
          title = "Proportion of users with at least one reverted edit",
           caption = "Limited to published new content edits shown or eligible to be shown Paste Check")  +
    theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        axis.text.x = element_text(size = 24),
        axis.title.x = element_text(margin = margin(t = 20, unit = "pt")),
        legend.position= "none",
        axis.line = element_line(colour = "black")) 

      
p

There is a 12% relative decrease in the proportion of users that published at least one reverted edit; however, the sample size is small and this decrease represents just one less user in the test group that was reverted. Results are inconclusive.

By If multiple checks were shown

Code

paste_check_revert_byuser_bymultiple <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1
          & test_group == 'test (paste check shown)' 
         ) %>%
    group_by( multiple_checks_shown) %>%
    summarise(n_users = n_distinct(user_id),
              n_revert_users = n_distinct(user_id[was_reverted == 1])) %>% #limit to new content edits without a refernece
    mutate(revert_rate = paste0(round(n_revert_users/n_users * 100, 1), "%")) %>%   
    select(-c(2,3)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "Users with at least one reverted edit by if multiple checks were shown"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    multiple_checks_shown = "Multiple Check",
    #n_edits = "Number of published new content edits",
    #n_reverts = "Number of edits reverted ",
    revert_rate = "Proportion of new content edits that were reverted"
  ) %>%
    tab_source_note(
        gt::md('Limited to published new content edits shown or eligible to shown Paste Check')
    )


display_html(as_raw_html(paste_check_revert_byuser_bymultiple))

Users with at least one reverted edit by if multiple checks were shown
Multiple Check	Proportion of new content edits that were reverted
one paste check	10.1%
multiple paste checks	7.2%
Limited to published new content edits shown or eligible to shown Paste Check

By Platform

Code

paste_check_revert_byuser_byplatform <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1
         ) %>%
    group_by(platform, test_group) %>%
    summarise(n_users = n_distinct(user_id),
              n_revert_users = n_distinct(user_id[was_reverted == 1])) %>% #limit to new content edits without a refernece
    mutate(revert_rate = paste0(round(n_revert_users/n_users * 100, 1), "%")) %>%   
    #select(-c(2,3)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "Users with at least one reverted edit by platform"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    test_group = "Experiment Group",
    platform = "Platform",
    n_users = "Number of Users",
    n_revert_users = "Number of users reverted ",
    revert_rate = "Proportion of distinct users that were reverted"
  ) %>%
    tab_source_note(
        gt::md('Limited to users who published new content edits shown or eligible to shown Paste Check')
    )


display_html(as_raw_html(paste_check_revert_byuser_byplatform))

Users with at least one reverted edit by platform
Experiment Group	Number of Users	Number of users reverted	Proportion of distinct users that were reverted
mobile web
control (eligible but not shown paste check)	248	41	16.5%
test (paste check shown)	236	46	19.5%
desktop
control (eligible but not shown paste check)	1229	120	9.8%
test (paste check shown)	1431	115	8%
Limited to users who published new content edits shown or eligible to shown Paste Check

By User Experience

Code

paste_check_revert_byuser_byuserexp <- paste_check_publish_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1
         ) %>%
    group_by(experience_level_group, test_group) %>%
    summarise(n_users = n_distinct(user_id),
              n_revert_users = n_distinct(user_id[was_reverted == 1])) %>% #limit to new content edits without a refernece
    mutate(revert_rate = paste0(round(n_revert_users/n_users * 100, 1), "%")) %>%   
    #select(-c(2,3)) %>% # removing granular data columns for publication
    gt()  %>%
    tab_header(
    title = "Users with at least one reverted edit by user status"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    test_group = "Test group",
    experience_level_group = "User experience",
    n_users = "Number of users",
    n_revert_users = "Number of users reverted ",
    revert_rate = "Proportion of distinct users that were reverted"
  ) %>%
    tab_source_note(
        gt::md('Limited to users who published new content edits shown or eligible to shown Paste Check')
    )


display_html(as_raw_html(paste_check_revert_byuser_byuserexp))

Users with at least one reverted edit by user status
Test group	Number of users	Number of users reverted	Proportion of distinct users that were reverted
Unregistered
control (eligible but not shown paste check)	312	39	12.5%
test (paste check shown)	374	43	11.5%
Newcomer
control (eligible but not shown paste check)	329	34	10.3%
test (paste check shown)	381	34	8.9%
Junior Contributor
control (eligible but not shown paste check)	882	90	10.2%
test (paste check shown)	951	87	9.1%
Limited to users who published new content edits shown or eligible to shown Paste Check

Constructive Retention Rate (Second Week)

Hypothesis: Newcomers and Junior Contributors will be more likely to return to publish a new content edit in the future that does not include copyright violations because Paste Check will have caused them to realize when they are at risk of this not being true.

Methodology: First we reviewed the proportion of newcomers and Junior Contributors that publish an edit on a main namespace where Paste Check was shown and successfully return to make an unreverted edit to a main namespace 7 and 14 days after their first edit.

Code

# load  retention
constructive_retention_rate <-
  read.csv(
    file = 'data/constructive_retention_data_14day.tsv',  
    header = TRUE,
    sep = "\t",
    stringsAsFactors = FALSE
  )

Code

# Cleaning up dataset and renaming fields to clarify meanings

# Set experience level group and factor levels
constructive_retention_rate <- constructive_retention_rate %>%
  mutate(
    experience_level_group = case_when(
     user_edit_count == 0 & user_status == 'registered' ~ 'Newcomer',
     user_edit_count == 0 & user_status == 'unregistered' ~ 'Unregistered',
      user_edit_count > 0 &  user_edit_count <= 100 ~ "Junior Contributor",
      user_edit_count >  100 ~ "Non-Junior Contributor"   
    ),
    experience_level_group = factor(experience_level_group,
         levels = c("Unregistered","Newcomer", "Non-Junior Contributor", "Junior Contributor")
   ))  

#rename experiment field to clarify
constructive_retention_rate <- constructive_retention_rate %>%
  mutate(test_group = factor(test_group,
         levels = c('2025-09-editcheck-paste-control', '2025-09-editcheck-paste-test'),
         labels = c("control (no paste check)", "test (paste check available)")))


#rename platform from phone to mobile web to clarify meaning
constructive_retention_rate <-constructive_retention_rate %>%
  mutate(platform = factor(platform,
         levels = c('phone', 'desktop'),
         labels = c("mobile web", "desktop")))

Overall

Code


constructive_retention_overall <- constructive_retention_rate %>%
    group_by(test_group)  %>%
    summarise(return_editors = sum(return_editors),
              editors = sum(editors),
        retention_rate = paste0(round(return_editors/editors * 100, 1), "%"))

Code

constructive_retention_overall_table <- constructive_retention_overall  %>%
  gt()  %>%
  tab_header(
    title = "Constructive second week retention rate"
  )  %>%
  cols_label(
    test_group = "Experiment group",
    return_editors = "Number of editors that returned second week",
    editors = "Number of first week editors",
    retention_rate = "Retention rate"
  ) %>%
  opt_stylize(5) %>%
  tab_footnote(
    footnote = "Limited to users shown or eligible to be shown at least one paste check their first week",
    locations = cells_column_labels(
      columns = 'retention_rate'
    )
  ) 

display_html(as_raw_html(constructive_retention_overall_table))

Constructive second week retention rate
Experiment group	Number of editors that returned second week	Number of first week editors	Retention rate¹
control (no paste check)	346	4623	7.5%
test (paste check available)	323	4820	6.7%
¹ Limited to users shown or eligible to be shown at least one paste check their first week

There were no significant changes in the constructive edit rate for users shown Paste Check. Contributors in both the test and control groups had a second week retention rate of ~7% during the reviewed timeframe.

By platform

Code


constructive_retention_byplatform <- constructive_retention_rate %>%
    group_by(platform, test_group)  %>%
    summarise(return_editors = sum(return_editors),
              editors = sum(editors),
        retention_rate = paste0(round(return_editors/editors * 100, 1), "%"))

Code

constructive_retention_byplatform_table <- constructive_retention_byplatform %>%
  gt()  %>%
  tab_header(
    title = "Constructive second week retention rate by platform"
  )  %>%
  cols_label(
    test_group = "Experiment group",
    platform = "Platform",
    return_editors = "Number of editors that returned second week",
    editors = "Number of first week editors",
    retention_rate = "Retention rate"
  ) %>%
  opt_stylize(5) %>%
  tab_footnote(
    footnote = "Limited to users shown or eligible to be shown at least one paste check",
    locations = cells_column_labels(
      columns = 'retention_rate'
    )
  ) 

display_html(as_raw_html(constructive_retention_byplatform_table))

Constructive second week retention rate by platform
Experiment group	Number of editors that returned second week	Number of first week editors	Retention rate¹
mobile web
control (no paste check)	37	791	4.7%
test (paste check available)	41	796	5.2%
desktop
control (no paste check)	309	3832	8.1%
test (paste check available)	282	4024	7%
¹ Limited to users shown or eligible to be shown at least one paste check

By User Experience

Code


constructive_retention_byuserexp <- constructive_retention_rate %>%
    group_by(experience_level_group, test_group)  %>%
    summarise(return_editors = sum(return_editors),
              editors = sum(editors),
        retention_rate = paste0(round(return_editors/editors * 100, 1), "%"))

Code

constructive_retention_byuserexp_table <- constructive_retention_byuserexp %>%
  gt()  %>%
  tab_header(
    title = "Constructive second week retention rate by user experience"
  )  %>%
  cols_label(
    test_group = "Experiment group",
    experience_level_group = "Experience level group",
    return_editors = "Number of editors that returned second week",
    editors = "Number of first week editors",
    retention_rate = "Retention rate"
  ) %>%
  opt_stylize(5) %>%
  tab_footnote(
    footnote = "Limited to users shown or eligible to be shown at least one paste check",
    locations = cells_column_labels(
      columns = 'retention_rate'
    )
  ) 

display_html(as_raw_html(constructive_retention_byuserexp_table))

Constructive second week retention rate by user experience
Experiment group	Number of editors that returned second week	Number of first week editors	Retention rate¹
Unregistered
control (no paste check)	31	1165	2.7%
test (paste check available)	34	1230	2.8%
Newcomer
control (no paste check)	52	988	5.3%
test (paste check available)	45	1107	4.1%
Junior Contributor
control (no paste check)	263	2470	10.6%
test (paste check available)	244	2483	9.8%
¹ Limited to users shown or eligible to be shown at least one paste check

No statistically significant changes by user experience level; however, we do see a slightly higher decrease (-1.2 percentage points) in retention rate for Newcomers shown Paste Check. These are registered users making their first edit on the Wikipedia.

Constructive Retention Rate (Paste Check not shown again)

We also reviewed the proportion of newcomers and Junior Contributors that publish an edit Paste Check was activated within and return to make a new content edit where Paste Check was not shown 7 to 14 days after.

Code

# load  retention
retention_rate_nopaste <-
  read.csv(
    file = 'data/retention_nopaste_data.tsv',  
    header = TRUE,
    sep = "\t",
    stringsAsFactors = FALSE
  )

Code

# Cleaning up dataset and renaming fields to clarify meanings

# Set experience level group and factor levels
retention_rate_nopaste <- retention_rate_nopaste %>%
  mutate(
    experience_level_group = case_when(
     user_edit_count == 0 & user_status == 'registered' ~ 'Newcomer',
     user_edit_count == 0 & user_status == 'unregistered' ~ 'Unregistered',
      user_edit_count > 0 &  user_edit_count <= 100 ~ "Junior Contributor",
      user_edit_count >  100 ~ "Non-Junior Contributor"   
    ),
    experience_level_group = factor(experience_level_group,
         levels = c("Unregistered","Newcomer", "Non-Junior Contributor", "Junior Contributor")
   ))  

#rename experiment field to clarify
retention_rate_nopaste<- retention_rate_nopaste %>%
  mutate(test_group = factor(test_group,
         levels = c('2025-09-editcheck-paste-control', '2025-09-editcheck-paste-test'),
         labels = c("control (no paste check)", "test (paste check available)")))


#rename platform from phone to mobile web to clarify meaning
retention_rate_nopaste <-retention_rate_nopaste%>%
  mutate(platform = factor(platform,
         levels = c('phone', 'desktop'),
         labels = c("mobile web", "desktop")))

Overall

Code

retention_rate_nopaste_overall <- retention_rate_nopaste %>%
    group_by(test_group)  %>%
    summarise(return_editors = sum(return_editors),
              editors = sum(editors),
        retention_rate = paste0(round(return_editors/editors * 100, 1), "%"))

Code

retention_rate_nopaste_overall_table <- retention_rate_nopaste_overall  %>%
  gt()  %>%
  tab_header(
    title = "Constructive second week retention rate (no paste check shown)"
  )  %>%
  cols_label(
    test_group = "Experiment group",
    return_editors = "Number of editors that returned second week",
    editors = "Number of first week editors",
    retention_rate = "Retention rate"
  ) %>%
  opt_stylize(5) %>%
  tab_footnote(
    footnote = "Limited to users shown or eligible to be shown at least one paste check",
    locations = cells_column_labels(
      columns = 'retention_rate'
    )
  ) 

display_html(as_raw_html(retention_rate_nopaste_overall_table))

Constructive second week retention rate (no paste check shown)
Experiment group	Number of editors that returned second week	Number of first week editors	Retention rate¹
control (no paste check)	56	1564	3.6%
test (paste check available)	62	1747	3.5%
¹ Limited to users shown or eligible to be shown at least one paste check

There were also no changes in the constructive edit rate for users shown Paste Check, when we limit to users who were not shown or eligible to shown Paste Check on their return vist.

By platform

Code

retention_rate_nopaste_byplatform <- retention_rate_nopaste %>%
    group_by(platform, test_group)  %>%
    summarise(return_editors = sum(return_editors),
              editors = sum(editors),
        retention_rate = paste0(round(return_editors/editors * 100, 1), "%"))

Code

retention_rate_nopaste_byplatform_table <- retention_rate_nopaste_byplatform  %>%
  select(-c(3,4)) %>% # removing granular data columns for publication
  gt()  %>%
  tab_header(
    title = "Constructive second week retention rate (no paste check shown) by platform"
  )  %>%
  cols_label(
    test_group = "Experiment group",
    platform = "Platform",
    #return_editors = "Number of editors that returned second week",
    #editors = "Number of first week editors",
    retention_rate = "Retention rate"
  ) %>%
  opt_stylize(5) %>%
  tab_footnote(
    footnote = "Limited to users shown or eligible to be shown at least one paste check",
    locations = cells_column_labels(
      columns = 'retention_rate'
    )
  ) 

display_html(as_raw_html(retention_rate_nopaste_byplatform_table))

Constructive second week retention rate (no paste check shown) by platform
Experiment group	Retention rate¹
mobile web
control (no paste check)	1.6%
test (paste check available)	3.7%
desktop
control (no paste check)	4%
test (paste check available)	3.5%
¹ Limited to users shown or eligible to be shown at least one paste check

By User Experience

Code

retention_rate_nopaste_byuserexp <- retention_rate_nopaste %>%
    group_by(experience_level_group, test_group)  %>%
    summarise(return_editors = sum(return_editors),
              editors = sum(editors),
        retention_rate = paste0(round(return_editors/editors * 100, 1), "%"))

Code

retention_rate_nopaste_byuserexp_table <- retention_rate_nopaste_byuserexp  %>%
  select(-c(3,4)) %>% # removing granular data columns for publication
  gt()  %>%
  tab_header(
    title = "Constructive second week retention rate (no paste check shown) by user experience"
  )  %>%
  cols_label(
    test_group = "Experiment group",
    experience_level_group = "Experience level group",
    #return_editors = "Number of editors that returned second week",
    #editors = "Number of first week editors",
    retention_rate = "Retention rate"
  ) %>%
  opt_stylize(5) %>%
  tab_footnote(
    footnote = "Limited to users shown or eligible to be shown at least one paste check",
    locations = cells_column_labels(
      columns = 'retention_rate'
    )
  ) 

display_html(as_raw_html(retention_rate_nopaste_byuserexp_table))

Constructive second week retention rate (no paste check shown) by user experience
Experiment group	Retention rate¹
Unregistered
control (no paste check)	0.3%
test (paste check available)	2.3%
Newcomer
control (no paste check)	4%
test (paste check available)	2.5%
Junior Contributor
control (no paste check)	4.6%
test (paste check available)	4.5%
¹ Limited to users shown or eligible to be shown at least one paste check

Retention rates vary but user experience. We observed an increase in retention rate for unregistered users while we observed a decrease for newcomers and no change for Junior Contributors.

However, more data is needed to confirm if any of these changes are statistically significant.

Guardrail #1: Paste Check Decline Rate

Decription Proportion of published edits where a user declined a Paste Check prompt by indicating that it was irrelevant.

Methodology: We reviewed the proportion of published edits shown Paste Check wherein people elected to keep the text they added (i.e. the Paste Check was dismissed) and where the edit was not reverted within 48 hours.

This was determined by edits where the user dimissed a Paste Check at least once in a session (event.feature = 'editCheck-paste' AND event.action = 'action-keep'). The analysis includes splits by the reason the user selected for keeping the text.

Note: We do not have instrumentation available to know if the final published edit by these users still included pasted text. Some contributors may have dismissed the Paste Check but still revised their final text prior to publishing.

Code

# load data for assessing edit reject frequency
paste_check_reject_data <-
  read.csv(
    file = 'data/paste_check_rejects_data_ab.tsv',
    header = TRUE,
    sep = "\t",
    stringsAsFactors = FALSE
  )

Code

# Set experience level group and factor levels
paste_check_reject_data <- paste_check_reject_data %>%
  mutate(
    experience_level_group = case_when(
     user_edit_count == 0 & user_status == 'registered' ~ 'Newcomer',
     user_edit_count == 0 & user_status == 'unregistered' ~ 'Unregistered',
      user_edit_count > 0 &  user_edit_count <= 100 ~ "Junior Contributor",
      user_edit_count >  100 ~ "Non-Junior Contributor"   
    ),
    experience_level_group = factor(experience_level_group,
         levels = c("Unregistered","Newcomer", "Non-Junior Contributor", "Junior Contributor")
   ))  

#rename experiment field to clarify
paste_check_reject_data <- paste_check_reject_data %>%
  mutate(test_group = factor(test_group,
         levels = c('2025-09-editcheck-paste-control', '2025-09-editcheck-paste-test'),
         labels = c("control (no Paste Check)", "test (shown Paste Check)")))



#rename platform from phone to mobile web to clarify meaning
paste_check_reject_data <- paste_check_reject_data %>%
  mutate(platform = factor(platform,
         levels = c('phone', 'desktop'),
         labels = c("mobile web", "desktop")))

# rename Wiki names
paste_check_reject_data  <- paste_check_reject_data  %>%
  mutate(
    wiki = recode(wiki, !!!wiki_name_lookup)
  )

Code

#Set fields and factor levels to assess number of checks shown

paste_check_reject_data <- paste_check_reject_data %>%
  mutate(
    multiple_checks_shown = 
         ifelse(n_checks_shown > 1, "multiple checks shown", "single check shown"),  
     multiple_checks_shown = factor( multiple_checks_shown ,
         levels = c("single check shown", "multiple checks shown")))
         
# note these buckets can be adjusted as needed based on distribution of data
paste_check_reject_data <- paste_check_reject_data %>%
  mutate(
    checks_shown_bucket = case_when(
     is.na(n_checks_shown) ~ '0',
     n_checks_shown == 1  ~ '1', 
     n_checks_shown == 2 ~ '2',
     n_checks_shown > 2 & n_checks_shown <= 5 ~ "3-5",
     n_checks_shown > 5 & n_checks_shown <= 10 ~ "6-10", 
     n_checks_shown > 10 ~ "over 10" 
    ),
    checks_shown_bucket = factor(checks_shown_bucket ,
         levels = c("0","1","2", "3-5", "6-10", "over 10")
   ))

Code

# shorten and clarify reason field names
paste_check_reject_data <- paste_check_reject_data %>%
  mutate(
    reject_reason = case_when(
     reject_reason == 'no_reject_reason' ~ 'No reason provided',
     reject_reason == 'edit-check-feedback-reason-other'  ~ 'None applies', 
     reject_reason == 'edit-check-feedback-reason-wrote' ~ 'I wrote content',
     reject_reason == 'edit-check-feedback-reason-permission'  ~ 'I have permission'
    ),
    reject_reason = factor(reject_reason ,
         levels = c("No reason provided","None applies","I wrote content", "I have permission")
   ))

Overall

Code

# overall dismissal rate
paste_check_dismissal_overall <- paste_check_reject_data %>%
    filter(was_paste_check_shown == 1 & is_new_content == 1) %>% #limit to where shown
    summarise(n_edits = n_distinct(editing_session),
              n_rejects = n_distinct(editing_session[n_rejects > 0 & was_reverted == 0])) %>% # at least one paste check declined and edit not reverted
    mutate(dismissal_rate = paste0(round(n_rejects/n_edits * 100, 1), "%")) %>%   
    gt()  %>%
    tab_header(
    title = "Paste Check dismissal rate"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    n_edits = "Number of edits shown Paste check",
    n_rejects = "Number of edits that dimisssed Paste Check",
    dismissal_rate = "Proportion of edits where Paste Check was dismissed"
  ) %>%
    tab_source_note(
        gt::md('Limited to non-reverted published edits where at least one Paste Check was shown')
    )


display_html(as_raw_html(paste_check_dismissal_overall ))

Paste Check dismissal rate
Number of edits shown Paste check	Number of edits that dimisssed Paste Check	Proportion of edits where Paste Check was dismissed
2117	1160	54.8%
Limited to non-reverted published edits where at least one Paste Check was shown

Users elected to keep the pasted text when prompted at 54.8% of edits shown Paste Check.

This dismissal rate is similar to rates observed for Tone Check and slightly lower than rates observed for Reference Check.

Note: We do not have instrumentation available to know if the final published edit by these users still included pasted text. Some contributors may have dismissed the Paste Check but still revised their final text prior to publishing.

By dismissal reason

Code

paste_check_dismissal_byreason_overall <- paste_check_reject_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1 
           & n_rejects > 0 & was_reverted == 0) %>% #limit to where shown and user elected to keep text
    group_by(reject_reason)  %>%
    summarise(n_edits_rejected = n_distinct(editing_session)) %>%
    mutate(select_rate = paste0(round(n_edits_rejected/sum(n_edits_rejected) * 100, 1), "%"))

Code

# plot bar chart of reason selection
dodge <- position_dodge(width=0.9)

p <- paste_check_dismissal_byreason_overall  %>%
    ggplot(aes(x= reject_reason, y = n_edits_rejected/sum(n_edits_rejected))) +
    geom_col(position = 'dodge', fill = 'dodgerblue4') +
    scale_y_continuous(labels = scales::percent) +
      geom_text(aes(label = paste(select_rate, "\n", n_edits_rejected,"edits"), fontface=2), vjust=1.2, size = 10, color = "white") +
    scale_fill_manual(values= cbPalette, name = "Reason")  +
    labs (y = "Percent of edits ",
           x = "Selected reason",
          title = "Reasons users selected for keeping pasted text",
           caption = "Limited to non-reverted published edits where a user selected to keep pasted text")  +
   theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        axis.text.x = element_text(size = 24),
        axis.title.x = element_text(margin = margin(t = 20, unit = "pt")),
        legend.position= "none",
        axis.line = element_line(colour = "black")) 

      
p

Users selected “I wrote this content and its not published elsewhere” in a little over half (55%) of all unreverted published edits where the user selected to keep their pasted text.

We would need to review a sample of the final published edits further to confirm if users are selecting this option in error or if there is confusion for what this decline option means. The placement of this option in the feedback window may also impact how often it is selected compared to other options..

By if multiple checks were shown

Code


paste_check_dismissal_bymultiple <- paste_check_reject_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to where shown
    group_by(multiple_checks_shown) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_rejects = n_distinct(editing_session[n_rejects > 0 & was_reverted == 0])) %>% #limit to new content edits without a refernece
    mutate(dismissal_rate = paste0(round(n_rejects/n_edits * 100, 1), "%")) %>%   
    gt()  %>%
    tab_header(
    title = "Paste Check dismissal rate by if multiple checks shown"
      )  %>%
 opt_stylize(5) %>%
  cols_label(
     multiple_checks_shown = "Multiple Checks",
    n_edits = "Number of edits shown Paste Check",
    n_rejects = "Number of edits that dimisssed Paste Check",
    dismissal_rate = "Proportion of edits where Paste Check was dismissed"
  ) %>%
    tab_source_note(
        gt::md('Limited to non-reverted published edits where at least one Paste Check was shown')
    )

display_html(as_raw_html(paste_check_dismissal_bymultiple ))

Paste Check dismissal rate by if multiple checks shown
Multiple Checks	Number of edits shown Paste Check	Number of edits that dimisssed Paste Check	Proportion of edits where Paste Check was dismissed
single check shown	1412	679	48.1%
multiple checks shown	705	482	68.4%
Limited to non-reverted published edits where at least one Paste Check was shown

We see a higher dismissal rate if more Paste checks are shown in a single session, which is expected.

By platform

Code

paste_check_dismissal_byplatform <- paste_check_reject_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to where shown
    group_by(platform) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_rejects = n_distinct(editing_session[n_rejects > 0 & was_reverted == 0])) %>% #limit to new content edits without a refernece
    mutate(dismissal_rate = paste0(round(n_rejects/n_edits * 100, 1), "%")) %>% 
    ungroup() %>%
    #mutate(n_edits = ifelse(n_edits < 50, "<50", n_edits),
     #n_rejects = ifelse(n_rejects < 50, "<50", n_rejects))  %>% #sanitizing per data publication guidelines
    #select(-2) %>%
    gt()  %>%
    tab_header(
    title = "Paste Check dismissal rate by platform"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    platform = "Platform",
    n_edits = "Number of edits shown Paste check",
    n_rejects = "Number of edits that dimisssed Paste Check",
    dismissal_rate = "Proportion of edits where Paste Check was dismissed"
  ) %>%
    tab_source_note(
        gt::md('Limited to non-reverted published edits where at least one Paste Check was shown')
    )

display_html(as_raw_html(paste_check_dismissal_byplatform ))

Paste Check dismissal rate by platform
Platform	Number of edits shown Paste check	Number of edits that dimisssed Paste Check	Proportion of edits where Paste Check was dismissed
mobile web	285	135	47.4%
desktop	1832	1026	56%
Limited to non-reverted published edits where at least one Paste Check was shown

Users are more likely to keep their pasted text on desktop. Users selected to keep the pasted text at 47.4% of all published mobile web edits where Paste Check was shown compared to 56% of desktop published edits.

Dismissal reason by platform

Code

paste_check_dismissal_byreason_byplatform <- paste_check_reject_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1 & 
           n_rejects > 0 & was_reverted == 0) %>% #limit to where shown and user elected to keep text
    group_by(platform, reject_reason)  %>%
    summarise(n_edits_rejected = n_distinct(editing_session)) %>%
    mutate(select_rate = round(n_edits_rejected/sum(n_edits_rejected), 2))

Code

# plot bar chart of reason selection
dodge <- position_dodge(width=0.9)

p <- paste_check_dismissal_byreason_byplatform  %>%
    ggplot(aes(x= reject_reason, y =select_rate, fill = reject_reason)) +
    geom_col(position = 'dodge',) +
    scale_y_continuous(labels = scales::percent) +
      geom_text(aes(label = paste0(select_rate * 100, "%"), fontface=2), vjust=1.2, size = 10, color = "white") +
    facet_grid(~ platform ) +
    labs (y = "Percent of edits ",
           x = "Selected reason",
          title = "Reasons users selected for keeping pasted text")  +
      scale_fill_manual(values= cbPalette, name = "Reason")  +
   theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        legend.position= "bottom",
        axis.text.x = element_blank(),
        axis.ticks.x = element_blank(),
        axis.line = element_line(colour = "black")) 

      
p

“I wrote this content…” is the most frequently selected reason for keeping text on both platforms.

By User Experience

Code

paste_check_dismissal_byuserexp <- paste_check_reject_data %>%
    filter(is_new_content ==1 & was_paste_check_shown == 1) %>% #limit to where shown
    group_by(experience_level_group) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_rejects = n_distinct(editing_session[n_rejects > 0 & was_reverted == 0])) %>% #limit to new content edits without a refernece
    mutate(dismissal_rate = paste0(round(n_rejects/n_edits * 100, 1), "%")) %>%   
    ungroup() %>%
    mutate(n_edits = ifelse(n_edits < 50, "<50", n_edits),
     n_rejects = ifelse(n_rejects < 50, "<50", n_rejects))  %>% #sanitizing per data publication guidelines
    #select(-2) %>%
    gt()  %>%
    tab_header(
    title = "Paste Check dismissal rate by user experience"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    experience_level_group = "User Experience",
    n_edits = "Number of edits shown Paste check",
    n_rejects = "Number of edits that dimisssed Paste Check",
    dismissal_rate = "Proportion of edits where Paste Check was dismissed"
  )%>%
    tab_source_note(
        gt::md('Limited to non-reverted published edits where at least one Paste Check was shown')
    )

display_html(as_raw_html(paste_check_dismissal_byuserexp ))

Paste Check dismissal rate by user experience
User Experience	Number of edits shown Paste check	Number of edits that dimisssed Paste Check	Proportion of edits where Paste Check was dismissed
Unregistered	449	217	48.3%
Newcomer	391	227	58.1%
Junior Contributor	1277	718	56.2%
Limited to non-reverted published edits where at least one Paste Check was shown

Newcomers (users making their first edit on the Wikipedia) are dismissing Paste Check at slightly higher rates compared to unregistered users or Junior Contributors.

We observed a similar trend in the Tone Check leading indicator analysis.

Dismissal reason by user experience

Code

paste_check_dismissal_byreason_byuserexp <- paste_check_reject_data %>%
    filter(is_new_content ==1 & was_paste_check_shown == 1 
           & n_rejects > 0 & was_reverted == 0) %>% #limit to where shown and user elected to keep text
    group_by(experience_level_group, reject_reason)  %>%
    summarise(n_edits_rejected = n_distinct(editing_session)) %>%
    mutate(select_rate = round(n_edits_rejected/sum(n_edits_rejected),2))

Code

# plot bar chart of reason selection
dodge <- position_dodge(width=0.9)

p <- paste_check_dismissal_byreason_byuserexp %>%
    ggplot(aes(x= reject_reason, y = select_rate, fill = reject_reason)) +
    geom_col(position = 'dodge') +
    scale_y_continuous(labels = scales::percent) +
      geom_text(aes(label = paste0(select_rate * 100, "%"), fontface=2), vjust=1.2, size = 10, color = "white") +
    facet_grid( ~ experience_level_group) +
    labs (y = "Percent of edits ",
           x = "Selected reason",
          title = "Reasons users selected for keeping pasted text")  +
    scale_fill_manual(values= cbPalette, name = "Reason")  +
    theme(
        panel.grid.minor = element_blank(),
        panel.background = element_blank(),
        plot.title = element_text(hjust = 0.5),
        text = element_text(size=24),
        legend.position= "bottom",
        axis.text.x = element_blank(),
        axis.ticks.x = element_blank(),
        axis.line = element_line(colour = "black")) 
      
p

The selected reason varies based on user experience.

Registered newcomers (user making their first edit) selected the “I wrote this content…” option at 62% of all published edits where Paste Check was dismissed.

Unregistered users are the most likely to select “None of the above applies” or “I have permission to reuse” options.

By partner Wikipedia

Code

paste_check_dismissal_bywiki <- paste_check_reject_data %>%
    filter(is_new_content == 1 & was_paste_check_shown == 1) %>% #limit to where shown
    group_by(wiki) %>%
    summarise(n_edits = n_distinct(editing_session),
              n_rejects = n_distinct(editing_session[n_rejects > 0 & was_reverted == 0])) %>%
    mutate(dismissal_rate = paste0(round(n_rejects/n_edits * 100, 1), "%")) %>% 
    filter(n_edits > 50) %>%  # limit to wikis with over 50 edits.
   ungroup() %>%
    mutate(n_edits = ifelse(n_edits < 50, "<50", n_edits),
    n_rejects = ifelse(n_rejects < 50, "<50", n_rejects))  %>% #sanitizing per data publication guidelines
    select(-2) %>%
    gt()  %>%
    tab_header(
    title = "Paste Check dismissal rate by partner Wikipedia"
      )  %>%
    opt_stylize(5) %>%
  cols_label(
    wiki = "Wikipedia",
    #n_edits = "Number of edits shown Paste check",
    n_rejects = "Number of edits that dimisssed Paste Check",
    dismissal_rate = "Proportion of edits where Paste Check was dismissed"
  ) %>%
    tab_source_note(
        gt::md('Limited to Wikipedias where Paste Check was shown in at least 50 edits')
    )

display_html(as_raw_html(paste_check_dismissal_bywiki ))

Paste Check dismissal rate by partner Wikipedia
Wikipedia	Number of edits that dimisssed Paste Check	Proportion of edits where Paste Check was dismissed
Arabic Wikipedia	<50	54.8%
Catalan Wikipedia	57	50.9%
Chinese Wikipedia	61	62.9%
Czech Wikipedia	<50	60.3%
Dutch Wikipedia	65	65.7%
German Wikipedia	183	52.7%
Indonesian Wikipedia	57	53.8%
Italian Wikipedia	125	53.4%
Persian Wikipedia	59	46.5%
Polish Wikipedia	55	50.9%
Russian Wikipedia	244	58.4%
Simple English Wikipedia	<50	67.7%
Ukrainian Wikipedia	54	46.6%
Vietnamese Wikipedia	59	60.2%
Limited to Wikipedias where Paste Check was shown in at least 50 edits

Guardrail #2: Block Rate

Description Proportion of contributors blocked after publishing an edit where Paste Check was shown, compared to contributors eligible but not shown Paste Check.

Methodology: We gathered all edits where edit check was shown from the mediawiki_revision_change_tag table and joined with mediawiki_private_cu_changes to gather user name info. We then reviewed both global and local blocks made within 6 hours of the Paste Check event as identified in the logging table.

Code

# load data for assessing blocks
edit_check_blocks <-
  read.csv(
    file = 'data/paste_check_eligible_users_blocked.csv',
    header = TRUE,
    sep = ",",
    stringsAsFactors = FALSE
  )

Code

#rename experiment field to clarify
edit_check_blocks <- edit_check_blocks%>%
  mutate(test_group = factor(bucket,
         levels = c('2025-09-editcheck-paste-control', '2025-09-editcheck-paste-test'),
         labels = c("control (no Paste Check)", "test (Paste Check available)")))

Code

edit_check_local_blocks_overall <- edit_check_blocks %>%
    #filter(user_id == 0) %>% #filter to identify logged out users
    group_by(test_group) %>%
    summarise(blocked_users = n_distinct(ip[is_local_blocked == 'True' | is_global_blocked == 'True']),
              all_users = n_distinct(ip))  %>%  #look at blocks
    mutate(prop_blocks = paste0(round(blocked_users/all_users * 100, 1), "%")) %>%
    select(-c(2,3)) %>% #removing granular data columns 
    gt()  %>%
    tab_header(
    title = "Proportion of users blocked by experiment group"
      )  %>%
  opt_stylize(5) %>%
  cols_label(
    test_group = "Test Group",
    prop_blocks = "Proportion of users blocked"
  )  %>%
    tab_source_note(
        gt::md('Limited to users blocked 6 hours after publishing an edit where Paste Check was shown')
    )


display_html(as_raw_html(edit_check_local_blocks_overall))

Proportion of users blocked by experiment group
Test Group	Proportion of users blocked
test (Paste Check available)	0.5%
Limited to users blocked 6 hours after publishing an edit where Paste Check was shown

Only 0.5% of all users were blocked after publishing an edit where at least one Paste check was shown.
No global blocks were issued to any users that published an edit where at least one Paste Check was shown.

Reuse

CC BY-SA 4.0