Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a free and non-commercial open data platform and infrastructure that facilitates research on pragmatic trials. It shares clean trial data and ratings using PRECIS-2, permitting multiple and varied meta-epidemiological studies to compare treatment effects estimates across trials that have different levels of pragmatism and other design features.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. The term "pragmatic", however, is used inconsistently and its definition and assessment require further clarification. Pragmatic trials should be designed to guide clinical practice and policy decisions, not to confirm a physiological or clinical hypothesis. A pragmatic trial should try to be as close as it is to real-world clinical practices, including recruiting participants, setting, design, delivery and execution of interventions, determining and analysis outcomes, and primary analyses. This is a major difference between explanation-based trials, as described by Schwartz and Lellouch1, which are designed to test a hypothesis in a more thorough manner.
The trials that are truly practical should avoid attempting to blind participants or the clinicians in order to result in distortions in estimates of the effects of treatment. Pragmatic trials should also seek to recruit patients from a wide range of health care settings so that their results can be compared to the real world.
Finally studies that are pragmatic should focus on outcomes that are vital to patients, like quality of life or functional recovery. This is especially important for trials involving surgical procedures that are invasive or have potentially dangerous adverse events. The CRASH trial29 compared a 2-page report with an electronic monitoring system for patients in hospitals suffering from chronic cardiac failure. The catheter trial28, on the other hand, used symptomatic catheter associated urinary tract infections as its primary outcome.
In addition to these characteristics, pragmatic trials should minimize the trial's procedures and requirements for data collection to reduce costs. In the end, pragmatic trials should aim to make their results as relevant to actual clinical practices as possible. This can be accomplished by ensuring that their analysis is based on the intention to treat approach (as defined in CONSORT extensions).
Despite these requirements however, a large number of RCTs with features that defy the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This can result in misleading claims of pragmaticity, and the use of the term must be standardized. The creation of the PRECIS-2 tool, which provides an objective standard for assessing practical features is a good initial step.
Methods
In a pragmatic trial the goal is to inform policy or clinical decisions by demonstrating how the intervention can be implemented into routine care. Explanatory trials test hypotheses concerning the cause-effect relation within idealized settings. In this way, pragmatic trials could have less internal validity than explanation studies and be more susceptible to biases in their design, analysis, and conduct. Despite these limitations, pragmatic trials may be a valuable source of information for decision-making in healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, with scores ranging between 1 and 5 (very pragmatic). In this study, the areas of recruitment, organization and flexibility in delivery, flexibility in adherence, and follow-up received high scores. However, the principal outcome and the method of missing data were scored below the practical limit. This suggests that a trial could be designed with well-thought-out practical features, yet not damaging the quality.
It is difficult to determine the degree of pragmatism in a particular trial because pragmatism does not have a single attribute. Certain aspects of a research study can be more pragmatic than other. A trial's pragmatism could be affected by modifications to the protocol or the logistics during the trial. Additionally 36% of the 89 pragmatic trials discovered by Koppenaal et al were placebo-controlled or conducted prior to approval and a majority of them were single-center. Therefore, they aren't quite as typical and can only be called pragmatic if their sponsors are tolerant of the lack of blinding in such trials.
Another common aspect of pragmatic trials is that the researchers try to make their results more meaningful by analysing subgroups of the trial sample. This can lead to unbalanced analyses that have lower statistical power. This increases the chance of missing or misdetecting differences in the primary outcomes. In the case of the pragmatic studies that were included in this meta-analysis this was a serious issue since the secondary outcomes weren't adjusted for the differences in baseline covariates.
In addition practical trials can be a challenge in the collection and interpretation of safety data. This is due to the fact that adverse events tend to be self-reported and are susceptible to delays, errors or coding differences. It is essential to improve the quality and accuracy of the results in these trials.
Results
Although the definition of pragmatism does not require that all clinical trials are 100% pragmatist, there are benefits to including pragmatic components in trials. These include:
By including routine patients, the results of the trial can be translated more quickly into clinical practice. However, pragmatic studies can also have disadvantages. The right amount of heterogeneity for instance could help a study expand its findings to different patients or settings. However, the wrong type can decrease the sensitivity of the test and, consequently, lessen the power of a trial to detect small treatment effects.
Several studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 created a framework to distinguish between explanation-based trials that support a physiological or clinical hypothesis and pragmatic trials that inform the selection of appropriate treatments in the real-world clinical setting. Their framework comprised nine domains that were scored on a scale of 1 to 5, with 1 indicating more explanatory and 5 indicating more pragmatic. The domains included recruitment and setting up, the delivery of intervention, flex compliance and primary analysis.
The original PRECIS tool3 was based on a similar scale and domains. Koppenaal et al10 devised an adaptation to this assessment, dubbed the Pragmascope which was more user-friendly to use in systematic reviews. They discovered that pragmatic reviews scored higher in all domains, but scored lower in the primary analysis domain.
This distinction in the analysis domain that is primary could be explained by the fact that most pragmatic trials analyze their data in the intention to treat manner while some explanation trials do not. The overall score for pragmatic systematic reviews was lower when the domains of organisation, flexible delivery and follow-up were merged.
It is important to understand that a pragmatic trial does not necessarily mean a low-quality trial, and in fact there is an increasing number of clinical trials (as defined by MEDLINE search, but it is neither specific or sensitive) that use the term 'pragmatic' in their abstract or title. The use of these terms in abstracts and titles could suggest a greater awareness of the importance of pragmatism, but it is unclear whether this is manifested in the content of the articles.
Conclusions
As appreciation for the value of evidence from the real world becomes more commonplace and pragmatic trials have gained popularity in research. They are clinical trials that are randomized that compare real-world care alternatives instead of experimental treatments in development, they include populations of patients which are more closely resembling the ones who are treated in routine care, they use comparators that are used in routine practice (e.g., existing medications) and depend on the self-reporting of participants about outcomes. This approach can overcome the limitations of observational research, like the biases associated with the reliance on volunteers, as well as the insufficient availability and codes that vary in national registers.
Pragmatic trials have other advantages, including the ability to use existing data sources and a greater probability of detecting meaningful distinctions from traditional trials. However, pragmatic tests may be prone to limitations that undermine their effectiveness and generalizability. Participation rates in some trials could be lower than anticipated due to the health-promoting effect, financial incentives, or competition from other research studies. Practical trials are often restricted by the need to recruit participants quickly. Additionally certain pragmatic trials don't have controls to ensure that the observed differences aren't due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified RCTs published from 2022 to 2022 that self-described as pragmatic. The PRECIS-2 tool was employed to determine the pragmatism of these trials. It covers domains such as eligibility criteria and flexibility in recruitment, adherence to intervention, and follow-up. They found 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or higher) in at least one of these domains.
프라그마틱 with a high pragmatism score tend to have broader eligibility criteria than traditional RCTs which have very specific criteria that are not likely to be present in the clinical setting, and comprise patients from a wide range of hospitals. The authors argue that these characteristics can help make the pragmatic trials more relevant and relevant to everyday clinical practice, however they do not necessarily guarantee that a trial conducted in a pragmatic manner is completely free of bias. The pragmatism is not a fixed attribute the test that doesn't have all the characteristics of an explicative study could still yield reliable and beneficial results.