We have addressed all reviewers comments as described below. We have no potential conflict of interest to report.

Transcription

1 Author s response to reviews Title: A web-based data visualization tool for the MIMIC-II database Authors: Joon Lee (joon.lee@uwaterloo.ca) Evan Ribey (eeribey@uwaterloo.ca) James Wallace (james.wallace@uwaterloo.ca) Version: 1 Date: 20 Jan 2016 Author s response to reviews: Editor: I have now received two reviews and I have also read the manuscript with interest. The reviewers have both suggested minor revisions, and one has additionally suggested that you consider including a "real-world example" to enhance the value of the research. Please make sure that you consider and respond to each of the reviewers' concerns and suggestions individually. Please also ensure that all potential conflicts of interest that might exist, including non-financial ones, are fully described. Response: We would like to thank the Editor for reading the manuscript and administering the peer review. As per Reviewer #2 s suggestion, the examples featured in Figures 2, 3, and 4 are now discussed in depth as real-world applications. Please see the second paragraph of the possible applications section. We have addressed all reviewers comments as described below. We have no potential conflict of interest to report. I would have liked to have seen a clearer description of the strengths and weaknesses (including limitations but also the functionality that may not yet exist but would be useful/needed to achieve the main aims) in the manuscript and a related statement in the abstract, if there is enough room. Response: We feel that what the current version of the tool can and cannot do have been described in detail in several places in the manuscript (the last paragraph of the background section, the possible applications section, the limitations section, and the future work section).

2 Many of these sections have been substantially strengthened while addressing the reviewers comments. The conclusions section of the abstract has been revised to clearly state the most important strengths and weaknesses of the tool. Reviewer #2: BMC-MIMIC-II The authors describe a visualization tool which allows researchers to explore the MIMIC-II database according to some basic dataset elements. This will help researchers determine feasibility for subsequent research studies - which will still require more advanced computer programming skills. The paper is well-written, but would benefit from inclusion of a realworld example so that the reader can visualize how this tool may help them vet a research question. Response: First of all, we would like to thank the reviewer for providing constructive comments and suggestions. As per your suggestion below, the examples featured in Figures 2, 3, and 4, are now discussed in depth as real-world examples. Please see our response below for details. ABSTRACT - Change "clinicians who have zero to limited..." to "clinicians who may have limited...", since it is unlikely these stakeholders have zero computer proficiency in the current era. - Suggest moving the first sentence of the results to the conclusions section. Since the first sentence of the Conclusions section is redundant, it was removed. BACKGROUND - Line 48: use the [3] reference, but also state the last name of the first author when referring to the publication.

3 - Line 53: remove "e.g.," prior to the reference numbers Line 59: Similar to the comment for the abstract, change "researchers who lack computer skills..." to "researchers who may lack the relevant computer programming skills..." - Lines 67-69: Your extension is to allow researchers to explore the feasibility of a potential study question, but does not allow researchers to rapidly become familiar with the MIMIC- II. The reviewer suggests saving this language for "future directions" in the discussion section. Response: Agreed. Since rapidly becoming familiar with MIMIC-II is a bit ambiguous, we toned down the language and changed rapidly become familiar with to rapidly explore. - Line 73: suggest removing "full-blown". IMPLEMENTATION - Time is an important dimension to researchers, particularly because health services delivery and medical innovations change over time. Does your "explore" feature include a time dimension? If not, please state in the limitations section of the discussion. Response: This is a great point. It is not possible for the tool to take time into account since MIMIC-II has been de-identified by randomly shifting all actual dates for public release. This issue has been mentioned in the second last paragraph of the limitation section. - Similarly, please address the exclusion of neonate data from your tool's functionality in your discussion. Pediatric researchers will not be able to vet their research questions using your vizualization tool.

4 Response: Agreed. This has been mentioned in the last paragraph of the limitation section. Also, including NICU data has been added to the last paragraph of the future work section. - Line 105: Suggest removing "in a nutshell". - How are missing data handled/presented in the resulting visualization? Response: All missing data are excluded. This has been clarified in the last paragraph of the main visualization features section. RESULTS AND DISCUSSION - Of interest to the reader is the total number of adults in the dataset and their associated demographics, as presented in your example. Perhaps this information could be presented in a table, and the example presented in the text could be a "real-world" application which explores the feasibility of a specific study question. In the reviewer's opinion, this would better demonstrate the usefulness of the tool's "explore" and "compare" funtions to the stakeholders involved. Response: The total number of unique adult patients in MIMIC-II is mentioned in the first paragraph of the main visualization features section. The adult patient characteristics of MIMIC-II have already been described in another publication ([2], Table 2), and the first paragraph of the background section already mentions that the descriptive statistics can be found in [2]. Since the focus of the present manuscript is on the data visualization tool rather than the MIMIC-II database, we feel that repeating the descriptive statistics reported in [2] is unnecessary. It is a great idea to pitch the existing examples as real-world applications. The second paragraph of the possible applications section now discusses in detail the two examples shown in Figures 2, 3, and 4 in the context of a precursor step for a research study. - Line 168: suggest removing "full-blown".

5 - Line 182: Instead of "brainstorm research ideas", use "screen for feasibility of research ideas", as the tool won't help researchers think of ideas. - Line 200: Change "multivariate" to "multivariable". The first implies a multi-level outcome, while the second refers to a model with multiple predictor variables. Reviewer #3: In "A web-based data visualization tool for the MIMIC-II database," the authors describe a simple web portal for the exploration and comparison of a subset of data from the MIMIC-II database. This manuscript is very well-written and easy to understand, and addresses a compelling use case. As a clinician who has worked with MIMIC-II, it was necessary for me to learn the intricacies of SQL as well as the MIMIC-II schema in order to accomplish the data extractions that I was interested in. While this was part of my goal in using MIMIC-II, many clinicians would not want to spend the time necessary to become familiar with SQL or other aspects of data mining. Thus, a web portal with a friendly UI is a welcome addition for MIMIC-II users. My comments are minor and should be easily addressed: >>The current version of MIMIC is MIMIC-III. Do you have plans to expand your visualization tool to include this expanded database? Response: First of all, we would like to thank the reviewer for providing constructive comments and suggestions. This is a great suggestion and the last paragraph of the future work section now discusses this. >>One notable category that is missing from your search options is medications prescribed. Given that there is no medication administration record (MAR) in MIMIC-II, the next-best-thing is the provider order entry (POE) for medications. We have used this information in several studies and have found it to be very informative. This data does need to be cleaned somewhat, as for example a medication might be present in the database in both generic and branded name format. We have recently published our algorithms that we used to normalize the POE data

6 which may be of interest: Response: This is a great suggestion. We have added this point to the second last paragraph of the future work section. The reviewer s paper has been cited as well. >>p.3 line 38 "as opposed to the waveforms" - please explain this a bit more for a reader who might not understand what these waveforms are (i.e., physiologic tracings gathered from the output of ICU monitors) >>p.3 line 40 "via two ways" - there are actually three ways to access MIMIC-II, the 3rd is through a download of the raw tables and a schema builder to manually construct the relational database. This would of course not be recommended for someone without a thorough knowledge of SQL and other computer software! Response: Agreed. Because this third option is less frequently used, we forgot to mention it. This option is now described in this paragraph. >>p.4 line 47: "The four ICU service types" - there are actually five, including the NICU - which you explain several paragraphs later when you say that your restricted to adults, only. Flows better if you move that exclusion criteria up a bit so that this sentence would be the four adult ICU service types >>p.8 line 55: "the tool can take up to a few minutes to complete" - true when the query is complex, but actually I tested out a few simple scenarios and the results came back very quickly. It might be more judicious to default the selections in the various sections to off, as a casual user is unlikely to need all information from every subsection. Response: Based on our consultation with first-time MIMIC users, most of them actually prefer to have all variables shown to them for initial exploration since they don t know what to expect. Also, many people tend to click the plot button right away without changing the default selections. If nothing is selected by default, the returned results are empty and the user may think that the tool doesn t work. Having said this, based

7 on our testing, the tool returns results within a minute even when all variables are selected. Hence, we changed up to a few minutes to up to a minute.