Types of studies

We included RCTs, as specified in the published protocol for this review (Cheng 2014).

Authors of a study reported in 2009 indicated that most of the reports of RCTs in Chinese journals lacked an adequate description of randomization, and that more than 90% of them did not adhere to accepted methodology for randomization (Wu 2009). Based on a more recent study, only 1 of 14 Chinese language studies labeled as RCTs could be confirmed to be a controlled trial (Adams 2012). We thus contacted the authors of RCTs published in Chinese journals that claimed to have randomised participants but did not report details about the randomization methods in order to determine whether they used appropriate methods for random sequence generation. The interview questions were administered over the telephone or email and adapted from the survey developed by Wu 2009. We also contacted the authors of non‐Chinese language reports of studies labeled as RCTs whenever their reports did not include details about randomization methods.

Types of participants

We included RCTs that had enrolled participants with acute hordeola, including acute internal hordeola and acute external hordeola (styes). We excluded RCTs of people with chronic hordeola or chalazia.

Types of interventions

We included RCTs evaluating acupuncture, a key component of TCM. Acupuncture methods used for treating acute hordeola include but are not limited to body needle insertion on acupoints; bloodletting at Erjian (EX‐HN6) at the ear apex or at Taiyang (EX‐HN5) at the temporal region; pricking or bloodletting at Arshi points (i.e. reflection spots that are small, pink pustules or papules) at the regions of the thoracic vertebra between the scapulae; acupressure; and electroacupuncture. We excluded RCTs of laser acupuncture and noninvasive electrostimulation, as these interventions do not involve the mechanical stimulation of acupoints. We excluded RCTs of injection acupuncture, which involves the injection of a herb, drug, or vitamin at the acupuncture point, because the effect may result from the herb, drug, or vitamin rather than the mechanical stimulation of acupuncture. However, as acupuncture is often accompanied by moxibustion (a TCM therapy that consists of warming specific points on the body by burning dried mugwort (moxa)), we included RCTs using moxibustion as a co‐intervention with acupuncture.

We included RCTs that compared acupuncture with no treatment (observation), sham acupuncture, or an active conventional treatment (e.g. hot or warm compresses, lid scrubs, antibiotics, or steroids). We also included RCTs that compared acupuncture plus another treatment versus that treatment alone. The adjuvant treatments could be either those used in Western medicine (e.g. antibiotics) or TCM (e.g. intake of traditional Chinese herbal medicine), as long as the adjuvant treatment was given to both the acupuncture and control groups.

Types of outcome measures

Electronic searches

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) (which contains the Cochrane Eyes and Vision Trials Register 2016, Issue 6), Ovid MEDLINE, Ovid MEDLINE In‐Process and Other Non‐Indexed Citations, Ovid MEDLINE Daily, Ovid OLDMEDLINE (January 1946 to June 2016), Embase (January 1980 to June 2016), Latin American and Caribbean Health Sciences Literature Database (LILACS) (1982 to June 2016), PubMed (1948 to June 2016), the metaRegister of Controlled Trials (mRCT) (www.controlled-trials.com) (last searched May 2014), ClinicalTrials.gov (www.clinicaltrials.gov), and the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) (www.who.int/ictrp/search/en). We also searched the Chinese databases SinoMed (previously called the Chinese Biomedical Database, www.sinomed.ac.cn/), Chinese National Knowledge Infrastructure (www.cnki.net/), and VIP Database for Chinese Technical Periodicals (lib.cqvip.com/). We did not use any date or language restrictions in the electronic searches for trials. We last searched the electronic databases on 7 June 2016.

See: Appendices for details of search strategies for CENTRAL (Appendix 1), MEDLINE (Appendix 2), Embase (Appendix 3), LILACS (Appendix 4), PubMed (Appendix 5), Chinese CNKI Database (Appendix 6), SinoMed Database (Appendix 7), Chinese VIP Database (Appendix 8), mRCT (Appendix 9), ClinicalTrials.gov (Appendix 10), and the ICTRP (Appendix 11).

Searching other resources

We reviewed the reference lists from potentially eligible studies to identify additional RCTs. We sought to obtain information about any ongoing study by contacting the study investigators.

Selection of studies

Two review authors independently screened all records identified by the electronic and other searches. According to the inclusion criteria, each review author assessed titles and abstracts as 'definitely relevant,' 'possibly relevant,' or 'definitely not relevant.' We resolved any discrepancies by discussion and then obtained the full‐text reports assessed by each author as 'definitely relevant' or 'possibly relevant.' Two review authors independently labeled each study as 'include,' 'awaiting classification,' or 'exclude.' For studies classified as 'awaiting classification,' we requested additional information from study investigators to make the judgement. We resolved any disagreements by discussion and documented the reasons for exclusion of studies in the Characteristics of excluded studies table. We gave study investigators two weeks to respond to our requests and documented any communication we had with them. If we did not hear back from the study investigators within two weeks, we used the available information to make our judgement.

Data extraction and management

Two review authors independently extracted data using the data extraction forms created by Cochrane Eyes and Vision. We pilot‐tested the extraction forms on a sample of included RCTs prior to the full extraction in order to ensure a similar extraction approach across review team members. For each included RCT, we extracted data on study design, participant characteristics, interventions, outcomes, and other relevant information. One review author entered the data into Review Manager 5 (RevMan 2014), and a second review author verified the data entry. Discrepancies between review authors were resolved by discussion.

When reported data were incomplete or ambiguous, we emailed or called the study author to request additional information or clarification. Whenever we did not receive a response in two weeks, we included the data as available in the study report.

Assessment of risk of bias in included studies

Two review authors independently assessed the risk of bias in the included RCTs based on the methods provided in Chapter 8 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011a). We used the following seven separate criteria:

• sequence generation;

• allocation concealment before randomization;

• masking of participants and study personnel;

• masking of outcome assessors;

• incomplete outcome data and percentage of missing data among treatment groups;

• selective outcome reporting; and

• other potential sources of bias.

For 'other potential sources of bias,' we assessed the following items:

• groups similar at baseline with regard to the most important prognostic indicators;

• co‐interventions avoided or similar between intervention groups;

• timing of the outcome assessment similar in all intervention groups;

• funding sources and conflicts of interest; and

• acceptable compliance with standards of all interventions.

We judged compliance to be acceptable when participants in the acupuncture group had received at least one session of acupuncture, as our two acupuncture adequacy assessors judged one session of acupuncture treatment to be enough for acute hordeolum. Our two acupuncture adequacy assessors judged that if the participants in the conventional group had used antibiotic or warm compresses daily for three days, this would be acceptable compliance, as they considered three days of conventional treatment to be common and enough for acute hordeolum.

We assessed the risk of bias for different outcomes based on whether masking of outcome assessors was performed and whether complete outcome data were consistently reported. For each RCT included in the review, we judged each criterion to have 'low,' 'unclear,' or 'high' risk of bias. When the information available in the published RCT reports was inadequate to assess risk of bias, we contacted the study investigators for clarification. If they did not respond within two weeks, we classified the RCT on the basis of the available information. We resolved discrepancies through discussion and by consulting a third review author when necessary.

Measures of treatment effect

Data analyses followed guidelines set forth in Chapter 9 of the Cochrane Handbook for Systematic Reviews of Interventions (Deeks 2011). We presented dichotomous data as risk ratios with 95% confidence intervals. We reported outcomes based on the follow‐up times in each RCT. Planned dichotomous outcome measures included the proportion of participants with the following outcomes: complete early and late resolution of a hordeolum; requirement for surgical incision and drainage after treatment; development of a chalazion after treatment; development of recurrent hordeola or secondary hordeola. When the RCT investigators reported improvement of acute hordeolum on an ordinal scale, such as 'resolved,' 'improved,' or 'ineffective,' we categorized the 'improved' and 'ineffective' groups as 'unresolved,' thus creating a dichotomous measure, 'resolved' and 'unresolved.' Planned continuous outcome measures included economic outcomes when data were available.

Unit of analysis issues

The unit of analysis for this review was the individual participant. The principle of acupuncture treatment for an acute hordeolum is to clear pathogenic "wind heat" in the body, which is a systemic effect and usually not limited to a specific eye (Cheng 2010; Shen 2007). We excluded any RCT in which participants had one eye randomised to acupuncture and the other eye to control.

Dealing with missing data

We conducted analyses by including RCTs with missing data according to the guidelines in Chapter 16 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011b). We conducted the primary analysis based on the data as reported. Whenever information was missing or unclear in the report, we contacted the primary study investigators. If they did not respond within two weeks, we used the data available in the study reports and described the limitations of this method when applicable. We did not impute data in primary analysis.

We imputed data in sensitivity analysis to assess how sensitive results were to reasonable changes in the assumptions being made (see Sensitivity analysis).

Assessment of heterogeneity

We evaluated clinical and methodological homogeneity based on similarity of study characteristics, participant inclusion/exclusion criteria, and primary and secondary outcomes. We assessed statistical heterogeneity using the I² statistic and considered an I² statistic of 50% or more as indicating substantial statistical heterogeneity. We also examined the Chi² statistic for heterogeneity and the degree of overlap in confidence intervals of included studies. Poor overlap suggests the presence of heterogeneity.

Assessment of reporting biases

We did not generate a funnel plot to assess the potential for publication bias, since we included fewer than 10 RCTs in the meta‐analysis. We assessed the possibility of selective outcome reporting for each study as part of the 'Risk of bias' assessment.

Data synthesis

When there was no important clinical or methodological heterogeneity among studies, we summarized the results of the studies in meta‐analyses. We used a random‐effects model in each analysis because of the expected heterogeneity among the studies' acupuncture protocols and settings. When there were data from fewer than three RCTs, we used the fixed‐effect model. We did not summarize results with meta‐analysis when substantial statistical heterogeneity (I² greater than 50%) was present, instead reporting individual RCT results only.

Subgroup analysis and investigation of heterogeneity

The type of hordeolum was the only prespecified characteristic for subgroup analysis. We did not conduct subgroup analysis due to lack of data. For the comparisons of acupuncture versus conventional treatments, each comparison included only one RCT, and none of the RCTs reported the outcomes separately by the type of hordeolum (Li 2006a; Xu 2004; Zhang 1991). For the comparison of acupuncture plus conventional treatments versus conventional treatments alone, one RCT reported the outcomes for internal hordeolum (Qi 2013), and two RCTs did not report the outcomes separately for each type of hordeolum (Pang 2009; Yang 2014).

Sensitivity analysis

If possible in future updates, we plan to conduct sensitivity analyses for meta‐analysis with greater than 10% of participants in the meta‐analysis had missing primary outcome data. We plan to impute data for the proportion of participants with complete resolution for the missing participants.

1. impute data using the proportion of participants with complete resolution observed in the control group;

2. impute data using the proportion of participants with complete resolution among completers in the separate groups (i.e. the acupuncture group and the control group, respectively); and

3. assume that all randomised participants with missing outcomes achieved a complete resolution (Higgins 2011b).

However, imputation does not reduce the risk of attrition bias, but we hope to assess if our meta‐analysis results might change using the above three methods of imputation. For our assessment of risk of attrition bias, please see Incomplete outcome data (attrition bias).

Also, due to the small number of included studies, we did not conduct planned sensitivity analyses to evaluate whether any meta‐analysis result that was statistically significant remained statistically significant when meta‐analysis was restricted only to RCTs judged to be at low risk of bias for the following domains: allocation concealment before randomization; masking of participants (for sham‐ versus non‐sham‐controlled trials); masking of outcome assessors; and completeness of outcome data. We planned to conduct sensitivity analyses to determine the impact of studies assessed as having inadequate treatment compliance by excluding RCTs judged as 'inadequate' and 'unclear' on each of the six aspects of adequacy of treatment.

Summary of findings

We have presented a 'Summary of findings' table of relative and absolute risks based on the risks across intervention groups from the included studies. Two review authors independently graded the overall certainty of the evidence for each outcome using the GRADE classification (www.gradeworkinggroup.org).

We assessed the certainty of evidence for each outcome as "high," "moderate," "low," or "very low" according to the following criteria as described in Chapter 12 of the Cochrane Handbook for Systematic Reviews of Interventions (Schünemann 2011).

1. High risk of bias among included studies.

2. Indirectness of evidence.

3. Unexplained heterogeneity or inconsistency of results.

4. Imprecision of results (i.e. wide confidence intervals).

5. High probability of publication bias.