Warning: mkdir(): Permission denied in /home/virtual/lib/view_data.php on line 87 Warning: chmod() expects exactly 2 parameters, 3 given in /home/virtual/lib/view_data.php on line 88 Warning: fopen(/home/virtual/neurointervention/journal/upload/ip_log/ip_log_2024-10.txt): failed to open stream: No such file or directory in /home/virtual/lib/view_data.php on line 95 Warning: fwrite() expects parameter 1 to be resource, boolean given in /home/virtual/lib/view_data.php on line 96 Protection of Personal Information in Medical Journal Publications

Protection of Personal Information in Medical Journal Publications

Article information

Neurointervention. 2019;14(1):1-8
Publication date (electronic) : 2019 February 19
doi : https://doi.org/10.5469/neuroint.2019.00031
Department of Parasitology and Institute of Medical Education, Hallym University College of Medicine, Chuncheon, Korea
Correspondence to: Sun Huh, MD Department of Parasitology and Institute of Medical Education, Hallym University College of Medicine, 1 Hallimdaehak-gil, Chuncheon 24252, Korea Tel: +82-33-248-2652 Fax: +82-33-241-1672 E-mail: shuh@hallym.ac.kr
Received 2019 January 20; Revised 2019 February 10; Accepted 2019 February 13.

Abstract

It aimed to present the definition of personal information based on Korean laws that protect personal information and the process of protection of personal information in journal publishing based on the guidelines of the International Committee of Medical Journal Editors and Committee of Publication Ethics. Two Korean laws relate to the protection of personal information in human subject research: the Personal Information Protection Act and the Bioethics and Safety Act. These laws were enacted to prevent the unauthorized use of Koreans’ personal information including medical information. Personal information can be divided into personally identifiable information including resident registration numbers and sensitive information including health information. To protect personal information in journal publishing, institutional review board (IRB) approval and obtaining informed consent from patients is recommended or mandatory in clinical studies. However, retrospective chart reviews may be exempted from IRB approval, while obtaining informed consent is recommended for all case reports. Journal policies may vary with regard to whether a copy of the informed consent form is collected from authors, since the Committee of Publication Ethics guideline does not specifically recommend collecting it. In discussions of adopting clinical data-sharing policies, transfer of data including nonidentifiable personal information to another country is an unresolved issue. Furthermore, a public data repository site should be established in Korea for data to be deposited. To protect subjects’ privacy and to prevent legal issues potentially arising from privacy concerns, editors and publishers should do their best to publish articles with appropriate oversight on subjects’ personal information.

INTRODUCTION

Protection of personal information in medical publishing is one of the most important topics in research and publication ethics, as it relates to the safety of subjects. Two Korean laws relate to the protection of personal information: the Personal Information Protection Act [1] and the Bioethics and Safety Act [2]. These laws were enacted to prevent the unauthorized use of Koreans’ personal information, including medical information. Therefore, as editors, we should adhere to the principles enshrined in these two laws and should be alert to these issues. This review article aimed to help medical editors and publishers process personal information appropriately according to the law and international guidelines. Specifically, I would like to explain the definition of personal information based on the Korean laws that protect personal information and the process of protection of personal information in medical journal publishing based on the guidelines of the International Committee of Medical Journal Editors (ICMJE) [3] and the Committee of Publication Ethics (COPE) [4]. In medical articles, personal information refers to the personal information of patients or study participants; therefore, in this review, these two terms were treated as the same concept.

DEFINITION OF PERSONAL INFORMATION BASED ON KOREAN LAW

According to Article 2 (Definitions) of the Personal Information Protection Act, the terms used in this act shall be defined as follows [1]: 1. The term “personal information” means information relating to a living individual that makes it possible to identify the individual by his/her full name, resident registration number, image, etc. (including information which, if not by itself, makes it possible to identify any specific individual if combined with other information); 2. The term “processing” means the collection, generation, connecting, interlocking, recording, storage, retention, value-added processing, editing, retrieval, output, correction, recovery, use, provision, and disclosure, destruction of personal information and other similar activities; 3. The term “data subject” means an individual who is identifiable by the information processed hereby to become the subject of that information; 4. The term “personal information file” means a set or sets of personal information arranged or organized in a systematic manner based on a certain rule for easy access to the personal information; 5. The term “personal information controller” means a public institution, legal person, organization, individual, etc. that processes personal information directly or indirectly to operate the personal information files for official or business purposes.

“Personal information,” as used above, includes two categories: personally identifiable information, which includes resident registration numbers, passport numbers, driver’s license numbers, and alien registration numbers, and sensitive information, which includes ideologies, beliefs, admission to or withdrawal from a trade union or political party, political opinions, health, sexual life, and other personal information that is likely to threaten the privacy of any data subject noticeably based on the Enforcement Decree of the Personal Information Protection Act [5]. However, if it is possible to identify an individual person based on a combination of sensitive information, it is treated as identifiable personal information. In contrast, sensitive information is not identifiable personal information if it is not possible to identify an individual person using that information. For example, combinations of information such as being a Buddhist, a member of the Democratic party, a diabetes patient who visits a clinic twice a year, married, an army veteran, and male cannot be identifiable personal information, because it is impossible to infer from that information who a specific individual person is, even though those items are categorized as sensitive information [1].

Recording and editing, which are included in the category of “processing,” are routine actions during journal publishing. The term “data subject” corresponds to patients or subjects in medical journals. The term “personal information file” corresponds to a patient’s medical record or a study subject’s characteristics and attributes. The term “personal information controller” refers not only to the researcher, author, or submitter of a manuscript, but also to the editor or publisher of a journal.

USE OF PERSONAL INFORMATION FOR RESEARCH PURPOSES

Personal information can be used for research purposes based on Article 18 of the Personal Information Protection Act, which is presented below:

“Article 18 (Limitation to Out-of-Purpose Use and Provision of Personal Information)

(1) A personal information controller shall not use personal information beyond the scope provided for in Article 15 (1), or provide it to any third party beyond the scope provided for in Article 17 (1) and (3).

(2) Notwithstanding paragraph (1), where any of the following subparagraphs applies, a personal information controller may use personal information or provide it to a third party for other purpose than the intended one, unless it is likely to infringe on unfairly the interest of a data subject or third party: ...

1. Where additional consent is obtained from the data subject; 2. Where special provisions exist in other laws; 3. Where it is deemed necessary explicitly for protecting, from impending danger, life, body or economic profits of the data subject or third party where the data subject or his/her legal representative is not in a position to express his/her intention, or prior consent cannot be obtained owing to unknown addresses; 4. Where personal information is provided in a manner keeping a specific individual unidentifiable necessarily for such purposes as compiling statistics or academic research.”

Therefore, obtaining informed consent from patients or subjects is required for the use of personal information in a human subject study. However, when personally identifiable information is not provided, the requirement to obtain informed consent may be waived for statistical or academic research. The latter case is relevant for retrospective chart reviews. If previous medical records extending back 10 years are reviewed and analyzed, it is nearly impossible to obtain informed consent from patients. If medical records are provided to researchers without personally identifiable information, it is nearly impossible to identify individual persons. However, if a study is a prospective cohort study or a randomized controlled study, researchers should consider obtaining informed consent from patients or subjects.

PROTECTION OF HUMAN SUBJECTS BASED ON THE BIOETHICS AND SAFETY ACT OF KOREAN GOVERNMENT

Human subjects of medical studies are protected by another law, the Bioethics and Safety Act [2].

Its terms are defined as follows: 1. The term “human subjects research project” means a research project specified by Ordinance of the Minister of Health and Welfare, such as a research project physically involving a human being as a subject, a research project conducted through communication, physical contact or other means of interaction, a research project conducted by using information with which individuals can be identified; 2. The term “human subject of research” means a person who is the subject of a human subject’s research project; ... 7. The term “personally identifiable information” means information with which an individual can be identified, such as the name, resident registration number, etc. of a human subject of research or the donor of an embryo, ovum, sperm, or human material (hereinafter referred to as “human subject of research or donor”); 18. The term “personal information” means information about an individual, such as personally identifiable information, genetic information, or information about health; 19. The term “anonymization” means the deletion of personally identifiable information permanently or full or partial substitution of personally identifiable information.

The definitions of terms in the Bioethics and Safety Act are somewhat different from those in the Personal Information Safety Act; however, they are similar conceptually.

The process through which human subjects are protected is presented below:

“Article 15 (Examination of Research on Human Subjects)

(1) A person who intends to conduct research on human subjects shall prepare a research plan and submit it for examination by the competent institutional committee before commencing research on human subjects.

(2) Notwithstanding paragraph (1), a research project may be exempted from examination by the competent institutional committee, if a risk to human subjects of research and the general public is insignificant and the research project meets the standards prescribed by Ordinance of the Ministry of Health and Welfare through deliberation by the National Committee.”

Exemption from IRB approval may be possible in research projects that meet the standards prescribed by Ordinance of the Ministry of Health and Welfare, according to Article 13, Enforcement Rule of the Bioethics and Safety Act [6].

“Research with publicly accessible data or research that does not involve collecting or recording personally identifiable information in the following cases: 1. Although subjects are manipulated directly or their environment is manipulated, they can be exempted first, if there is no drug administration, blood collection or other invasive procedure, and second, if the researcher uses a simple contact instrument or observation instrument....; 2. Research in which the subjects are not specified and no sensitive information is collected or recorded, even though researchers meet subjects directly; 3. Research with pre-existing data or documents on subjects.”

According to Article 2, Enforcement Rule of the Bioethics and Safety Act [6], below researches are not included as human subject study; therefore, IRB approval nor informed consent is required: 1. Research to review or evaluate the public welfare or public service program directly or commissioned by the Government or Local government. 2. Research on the educational practice in educational institutes designated based on Korean law.

ICMJE RECOMMENDATIONS FOR THE CONDUCT, REPORTING, EDITING, AND PUBLICATION OF SCHOLARLY WORK IN MEDICAL JOURNALS

This recommendation announced that “All authors should seek approval to conduct research from an independent local, regional or national review body before conducting and reporting of human research. ... Identifying information, including names, initials, or hospital numbers, should not be published in written descriptions, photographs, or pedigrees unless the information is essential for scientific purposes and the patient (or parent or guardian) gives written informed consent for publication.” [3]

COPE JOURNALS’ BEST PRACTICES FOR ENSURING CONSENT FOR PUBLISHING MEDICAL CASE REPORTS

The COPE guideline for personal information protection for case reports states: “Publication consent forms should be required for any case report in which an individual or a group of individuals can be identified. This requirement also applies when a report involves deceased persons. Examples of identifying information are descriptions of individual case histories, photos, x-rays, or genetic pedigrees.” It further specifies that “journals should not themselves collect the signed consent forms, because the receipt and storage of confidential patient information could subject them to cumbersome security requirements and potential legal liability.” [4]

A CASE STUDY OF THE PROTECTION OF PERSONAL INFORMATION: NEUROINTERVENTION

The above laws and guidelines are generally adopted by medical journals throughout the world. In particular, most medical journals in Korea have adopted the ICMJE recommendations. Here, I would like to discuss some issues originating from the protection of personal information based on an example from Neurointervention, a small but very unique journal in its field. Neurointervention follows the ICMJE recommendation for “protection of research participants,” as reflected in the following ethical statement: [7]

“Manuscripts involving examinations of volunteers and patients must include a statement that the trial protocol has been approved by an IRB and that the subjects gave informed consent thus meets the standards of the Declaration of Helsinki in its revised version that the World Medical Association has developed as a statement of ethical principles for medical research involving human subjects, including research on identifiable human material and data (http://www.wma.net). Clinical studies that do not meet the Helsinki Declaration will not be considered for publication. Human subjects should not be identifiable, such that patient’s names, initials, hospital numbers, dates of birth, or other protected healthcare information should not be disclosed.”

I analyzed the articles published in Neurointervention in 2018 to determine whether all articles fulfilled the above statement. The publication type, study design, subjects, approval by institutional review board, and informed consent were analyzed in 11 original articles, seven case reports, one review, and one brief report. The data are presented in Table 1. Sixteen of the 20 citable articles contained human subjects. Of those 16 articles, six were retrospective chart reviews, of which three received IRB approval and one mentioned that “no approval was necessary.” There were two prospective cohort studies, of which one received IRB approval and the other contained no statement about IRB approval. There were seven case reports, of which only two stated that the researchers obtained informed consent from patients. There were no statements of informed consent in the other five case reports. One review article did not contain a statement about whether informed consent was obtained, although it presented findings of cerebral angiography and magnetic resonance imaging (MRI). The above findings led us to ask some questions.

IRB approval and status of whether informed consent was obtained among articles published in Neurointervention in 2018

Question 1: is it mandatory to receive IRB approval for retrospective chart reviews?

According to the Bioethics and Safety Act of the Korean government [2], research into human subjects should be approved by the IRB of the researchers’ institution as follows: “‘Human subjects research project’ means a research project specified by Ordinance of the Minister of Health and Welfare, such as a research project physically involving a human being as a subject, a research project conducted through communication, physical contact or other means of interaction, a research project conducted by using information with which individuals can be identified.” However, exemption from IRB approval may be possible according to Article 13, Enforcement Rule of the Bioethics and Safety Act as described above [6]. The Office of Human Research Protections of the US Department of Health & Human Services states that retrospective chart reviews are eligible for IRB exemption [8]. The Human Subject Regulations Decision Charts for retrospective chart reviews are shown in Fig. 1. The ICMJE has also presented the following statement: “All investigators should ensure that the planning conduct and reporting of human research are in accordance with the Helsinki Declaration as revised in 2013. All authors should seek approval to conduct research from an independent local, regional or national review body (e.g., ethics committee, institutional review board).” [4]

Fig. 1.

Diagram of the institutional review board exemption process of retrospective chart reviews, adapted from chart 5: Does Exemption 45 CFR 46.101(b)(4) (for Existing Data, Documents, Records and Specimens) Apply? Modified from International Committee of Medical Journal Editors[8].

Based on the above law and guidelines, retrospective chart reviews in which subjects cannot be identified may be submitted to the IRB for exemption. Furthermore, informed consent may be waived. The authors of an article from Germany mentioned that “no approval was necessary”. However, all three articles from Korea received IRB approval and the requirement for informed consent from subjects was waived. Practices in this regard may vary from country to country. However, I recommend that researchers should submit their research proposal to the IRB for exemption and a waiver of informed consent, even for retrospective chart reviews. The decision of whether the data are nonidentifiable will be made by the IRB after reviewing the submitted proposal.

Question 2: is IRB approval mandatory for a prospective cohort study?

One of the two prospective cohort studies published in 2018 did not contain a statement indicating that it received IRB approval. Authors should receive approval for prospective cohort studies, and this should be checked before the review process.

Question 3: is it mandatory to obtain informed consent for all case reports?

Of the seven case reports published in Neurointervention in 2018, two reported that they had obtained informed consent. The figures of other five case studies showed only radiological findings such as MRI, MRI angiograms, computed tomography (CT), CT angiograms, angiography, and nasal endoscopic findings. It is nearly impossible to identify a patient based on those findings unless the patient or his/her family members are radiologists. These figures may be treated as nonidentifiable information. The ICMJE also stated that “identifying information, including names, initials, or hospital numbers, should not be published in written descriptions, photographs, or pedigrees unless the information is essential for scientific purposes and the patient (or parent or guardian) gives written informed consent for publication.” [3] In Neurointervention, the radiological photos contained no identifying information, including names, initials, or hospital numbers. However, the COPE guidance on best practice for ensuring consent for publishing medical case reports [4] includes X-rays as identifying information. Therefore, it is recommended to receive informed consent for every case report. If it is not possible to obtain informed consent from the patient or his/her family, it is recommended to receive IRB exemption. A medical journal published by a scholarly society in Korea was dropped from PubMed Central (PMC) in 2018 due to issues stemming from not obtaining informed consent in case reports in which photos of the trunk or faces with eye-masking were presented in the figures. This was the first instance where any of the 117 Korean scholarly journals indexed in PMC was excluded [9]. A similar case in another journal was handled without failure to meet the requirements of PMC. Corrigenda were published indicating that IRB approval was received and informed consent was obtained [10], and the background of the situation was explained in an editorial [11]. Recently, compliance with the Principles of Transparency and Best Practice in Journal Publishing (hereafter, best practice) [12] became mandatory for applications to MEDLINE and even to PMC. One of the best practice items relates to a “journal’s policy on ethical oversight.” Editors of medical journals from Korea strive to have their journals included in MEDLINE [13,14]; therefore, ethical issues relating to the protection of personal information should be resolved [15] before MEDLINE application.

Question 4: is it necessary for editors or publishers to collect copies of informed consent forms from authors?

Copies of informed consent forms for case reports may be received by the editor or publisher of some journals [3], while in other circumstances, they may not do so, but request that the authors archive those forms due to privacy issues [4]. This varies according to each journal’s policy. Some journals ask that researchers complete their own informed consent form, while others accept informed consent forms from authors’ institutions. This also depends on the journal’s policy.

Question 5: is it possible to adopt a data-sharing policy if the data include sensitive information?

Neurointervention has not yet adopted a clinical trial data-sharing policy, although all journal publishers and editors who follow the ICMJE guidelines should do so. The ICMJE announced that “Clinical trials that begin enrolling participants on or after 1 January 2019 must include a data sharing plan in the trial’s registration.” Therefore, it is necessary to determine whether it is possible to adopt such a policy based on Korean law. There is no problem with any medical journal adopting a clinical data-sharing policy according to which authors can decide whether they will share their data. If authors decide to share their data, the editor and publisher should carefully consider the following questions: 1) whether the policy is mandatory or optional for their journals (and an announcement should be made in either case); 2) how to handle the fact that the data should be nonidentifiable; 3) who will deposit the data (the author or editor) after a manuscript is accepted; 4) how long the data will be deposited; 5) what repository site will be used to archive the data; 6) what kind of data will be deposited; and 7) whether it is necessary for experts to review the data. In Korea, a journal (Journal of Educational Evaluation for Health Professions) adopted a data-sharing policy in which the editor deposits data to the Harvard Dataverse after accepting a manuscript without a specific limit to its duration. Furthermore, the entire dataset, ranging from raw data to analyzed data, is deposited with a review of the data [16]. There are no commercial or public repository sites for data deposition in Korea. It is time to establish those repository sites through the cooperation of editors and publishers, which will facilitate the adoption of data-sharing policies. If not, all data will be deposited to sites located in foreign countries.

Question 6: is it possible to deposit data to a repository in Korea or in other countries?

According to Korean law, it is not allowed to transfer personal data to foreign countries without consent from the data subject according to Article 17 (Provision of Personal Information), as follows: (3) A personal information controller shall inform a data subject of the matters provided for in paragraph (2), and obtain the consent from the data subject in order to provide personal information to a third party overseas; and shall not enter into a contract for the cross-border transfer of personal information in violation of this Act [1].

When an editor receives clinical data according to a data-sharing policy, the data may contain sensitive information, including health information. Consent to transfer data to a foreign country is not included in the informed consent form. It is possible to deposit data to a repository in Korea; however, if it is not possible to transfer data to a foreign country, researchers in Korea cannot submit a manuscript with clinical data to international journals that adopt data-sharing policy. Therefore, transferring such data should be allowed. Nonidentifiable data should be allowed to be transferred if sensitive information cannot be used to identify specific individuals, even through combinations of sensitive information. It is usually impossible to infer specific individuals by combining information if their identity is treated as anonymous.

There is no clear interpretation of the permissibility of data transfer to foreign countries, as the Korean government has not yet held an official debate or announcement, and no legal cases have established a relevant precedent. Therefore, this issue should be discussed in greater depth among researchers, personal information experts, government officers, and representatives because the propagation of medical information through journals is essential for the promotion of the people’s health. In the United States, among 771 participants of clinical trials, most (92%) respondents felt that the benefits of data sharing outweighed the potential negative consequences [17]. In Korea, no similar surveys have been conducted among the participants of clinical trials. Doing so would be an excellent step towards obtaining data from stakeholders on this issue.

CONCLUSION

In recent years, patients’ or subjects’ privacy has continually become more important. Concomitantly, the number of open access journals has increased, meaning that anyone with an internet connection can read medical articles without any barriers; furthermore, many people in the world can understand English-language articles through a variety of translation services. As such, all personal information in medical articles, including the photos, have been made widely available. Therefore, IRB approval for clinical studies and informed consent from subjects should be checked meticulously. Furthermore, establishing a data repository in Korea should be considered to encourage the deposition of nonidentifiable data in accordance with data-sharing policies adopted by journals. To protect subjects’ privacy and to prevent legal issues potentially arising from privacy concerns, editors and publishers should do their best to publish articles with appropriate oversight of patients’ or subjects’ personal information.

References

1. Ministry of Government Legislation of Korea. Personal information protection act. http://elaw.klri.re.kr/kor_service/lawView.do?lang=ENG&hseq=46731. Accessed January 19, 2019.
2. Ministry of Government Legislation of Korea. Bioethics and safety. http://elaw.klri.re.kr/kor_service/lawView.do?hseq=46341&lang=ENG. Accessed January 19, 2019.
3. International Committee of Medical Journal Editors. Protection of research participants. http://www.icmje.org/recommendations/browse/roles-and-responsibilities/protection-of-research-participants.html. Accessed January 20, 2019.
4. Committee of Publication Ethics. Journals’ best practice for ensuring consent for publishing medical case reports: guidance for COPE. https://publicationethics.org/resources/guidelines/journals%E2%80%99-best-practices-ensuring-consent-publishing-medical-case-reports. Accessed December 31, 2018.
5. Ministry of Government Legislation of Korea. Enforcement decree of the personal information protection act. http://elaw.klri.re.kr/kor_service/lawView.do?lang=ENG&hseq=45683. Accessed January 20, 2019.
6. Ministry of Government Legislation of Korea. Enforcement rule of bioethics and safety act. http://www.law.go.kr/LSW/lsInfoP.do?lsiSeq=206257#0000. Accessed January 20, 2019.
7. Korean Society of Interventional Neuroradiology. Research and publication ethics. https://neurointervention.org/authors/ethics.php. Accessed December 31, 2018.
8. Office of Human Research Protections. US Department of Health & Human Services. Human subject regulations decision charts. https://www.hhs.gov/ohrp/regulations-and-policy/decision-charts/index.html. Accessed December 31, 2018.
9. Huh S. Strengthened research ethics, including patient anonymity and informed consent, in MEDLINE and PubMed Central journals. Arch Craniofac Surg 2018;19:241–242.
10. Editorial Office, Archives of Plastic Surgery. Corrigenda: omission of the description of informed consent on the identifiable photos and the description on ethical treatment of experimental animals. Arch Plast Surg 2017;44:575–576.
11. Huh S. How to deal with ethical issues involving animal experiments and identifiable photographs in articles published in Archives of Plastic Surgery. Arch Plast Surg 2017;44:475–476.
12. Huh S. Adherence of the Annals of Pediatric Endocrinology and Metabolism to the Principles of Transparency and Best Practice in scholarly publishing. Ann Pediatr Endocrinol Metab 2018;23:1–3.
13. Huh S. How to add a journal to the international databases, Science Citation Index Expanded and MEDLINE. Arch Plast Surg 2016;43:487–490.
14. Huh S. How to prepare Endocrinology and Metabolism for reapplication to MEDLINE. Endocrinol Metab (Seoul) 2017;32:58–61.
15. Huh S. Ethical issue in preparing manuscript on esthetic patients. Arch Aesthetic Plast Surg 2014;20:1–2.
16. Huh S. Updates from 2018: being indexed in EMBASE, becoming an affiliated journal of the World Federation for Medical Education, implementing an optional open data policy, adopting principles of transparency and best practice in scholarly publishing, and appreciation to reviewers. J Educ Eval Health Prof 2018;15:36.
17. Mello MM, Lieou V, Goodman SN. Clinical trial participants’ views of the risks and benefits of data sharing. N Engl J Med 2018;378:2202–2211.

Article information Continued

Fig. 1.

Diagram of the institutional review board exemption process of retrospective chart reviews, adapted from chart 5: Does Exemption 45 CFR 46.101(b)(4) (for Existing Data, Documents, Records and Specimens) Apply? Modified from International Committee of Medical Journal Editors[8].

Table 1.

IRB approval and status of whether informed consent was obtained among articles published in Neurointervention in 2018

No. Issue Page Publication type Study design Subjects IRB approval Informed consent Case data
1 1 1 Original article Retrospective descriptive study Human Yes, without number Waived
2 1 13 Original article Retrospective descriptive study Human No No
3 1 20 Original article Retrospective analysis Human No No
4 1 32 Original article Retrospective study Human No approval was necessary
5 1 41 Original article Retrospective study Human Yes, without number Waived
6 1 48 Original article Retrospective study Human Yes, without number Waived
7 1 54 Case Case report Human No No CT angiogram, angiography
8 1 58 Case Case report Human No Yes CT, angiogram
9 1 62 Case Case report Human No No MRI, angiogram
10 1 66 Case Case report Human No No Angiogram, radiograph, nasal endoscopy
11 2 73 Review Literature review Human No No Angiography, MRI
12 2 84 Original article Electronic survey Hospital No No
13 2 90 Original article Systemic review Literature No No
14 2 100 Original article Meta-analysis Literature No No
15 2 110 Original article Prospective cohort Human Yes, with number Yes
16 2 117 Original article 3D model test Stent No No
17 2 124 Brief report Prospective cohort Human No No
18 2 129 Case Case report Human No Yes
19 2 133 Case Case report Human No No MR angiography
20 2 138 Case Case report Human No No MRI, angiogram

IRB, institutional review board; CT, computed tomography; MRI, magnetic resonance imaging; MR, magnetic resonance.