Call For Participation

Submissions are closed

Submissions to ML4H 2021 are now closed, but thank you for your interest in the event! Additional information on the event itself can be found here, and the full original Call for Participation is below.

Original Call

ML4H 2021 invites submissions describing innovative machine learning research focused on relevant problems in health and biomedicine. Authors are invited to submit works that fit anywhere within the broad purview of Machine Learning for Health. Similar to the last two years, ML4H 2021 will accept both papers for publication in a formal proceedings and non-archival extended abstract submissions.

In response to the growing ML4H community, for the first time, ML4H will be a separate symposium, rather than a NeurIPS workshop. This event represents a continuation of prior ML4H workshops (e.g. 2016, 2017, 2018, 2019, 2020) and will continue to be held in December shortly before NeurIPS. This will allow us to expand the reviewing timeline and introduce a formal author response and reviewer discussion period, which we hope will improve the reviewing experience.

ML4H 2021 will feature:

Important Dates

Sep 13th AoE: Submission Deadline

Oct 11th : Author Response Period Starts

Oct 15th : Author Response Period Ends

Oct 27th: Final Decisions Released

Nov 14th [tentative]: Camera Ready Deadline

Dec 4th: Virtual Event

Mentorship Opportunities

This year, ML4H is offering a Submission Mentorship Program which focuses on pairing less experienced authors with senior researchers to provide feedback on their paper submission, with the overall goal of improving submission quality and fostering future collaboration.

Please fill in the corresponding application form by August 3rd to participate in the second stage of the Submission Mentorship Program which will take place between Aug 5th and September 13th. More details about the Submission Mentorship Program can be found here.

Application form for mentees

Application form for mentors

ML4H will also be offering both a Reviewer Mentorship Program which will take place immediately after the submission deadline and its aim is to train junior reviewers, foster new connections and relationships in the ML4H community, and ultimately improve the quality of the review process. To express interest in participating in the Reviewer Mentorship Program, please fill out this form, and we will be in touch with additional details.

We especially encourage less experienced authors and reviewers and participants from underrepresented backgrounds to sign up as mentees, as well as more senior community members to serve as mentors for these programs.

Submission Tracks

Like last year, ML4H 2021 will feature two submission tracks: a full, archival proceedings track and a non-archival, extended abstract track. Submissions to either track will undergo double-blind peer review. It will be up to the authors to ensure the proper anonymization of their papers. Do not include any names or affiliations. Refer to your own past work in the third-person. Malformed, non-blinded, non-healthcare oriented, or grossly insufficient works may be desk rejected without undergoing additional review. In addition, submissions to both tracks will be featured at the event’s virtual poster session, and a subset of works (from either track) will be invited to give a spotlight presentation about their work.

Accepted papers and extended abstracts will be chosen based on their technical merit and contribution to the event. More details on how to write an excellent ML4H full paper or extended abstract can be found here.

Below are the salient differences between both tracks.

(A) Proceedings Track

Excellent ML4H Proceedings papers should be compelling, cohesive works with a high degree of technical sophistication as well as clear and high-impact relevance to healthcare. Accepted proceedings papers will be published in the Proceedings for Machine Learning Research (PMLR) . Full proceedings papers can be up to 9 pages (excluding references and appendices).

Papers that are submitted to the ML4H proceedings track cannot be already published or under review in any other archival venue. Similarly, papers published to the ML4H proceedings cannot be published again later at any other venue.

(B) Extended Abstract Track

An excellent extended abstract is one that leads to insight at the event through interaction with other attendees. This can be through presenting new ideas/ways of thinking, leading to insightful discussion and feedback, dissemination of new valuable resources, or enabling new opportunities for collaborations. We also especially solicit “non-traditional research artifacts” as submissions to the extended abstract track, such as papers highlighting novel datasets, insightful negative results, exciting preliminary results that warrant rapid dissemination, reproducibility studies, and opinion pieces or critiques.

Extended abstracts can be up to 4 pages (excluding references and appendices), though additional information not critical for understanding the work can be included in an appendix without penalty (reviewers will review the work based predominantly on the main text). Extended abstracts will not appear in the ML4H proceedings, but upon acceptance, we invite (but do not require) authors to submit their extended abstract to an ML4H arxiv.org index.

Authors of accepted extended abstracts (non-archival submissions) retain full copyright of their work, and acceptance of such a submission to ML4H 2021 does not preclude publication of the same material in another journal or conference. Furthermore, extended abstract submissions that are under review or have been recently published in a conference or a journal are allowed for submission. Authors should clearly state any overlapping published or submitted work at the time of submission, and must ensure that they are not violating any other venue dual submission policies.

Thematic Session

This year, we are hosting a thematic session on real-world robustness and generalization in machine learning for health. When submitting work to the symposium, authors will have the option to indicate whether or not their work fits within the theme. Any work within the broad purview of machine learning for health will be considered for ML4H, regardless of its suitability to the theme, and selecting this option will have no impact on overall acceptance to the symposium. However, papers submitted to the thematic submissions will be eligible for consideration for the Best Thematic Paper Award.

If machine learning systems are to be routinely used to support clinical decision making, it is critical that they reliably characterize and forecast patient health. However, machine learning models developed and evaluated with standard practices may underperform dramatically, unexpectedly, and inequitably when integrated into care processes. Furthermore, the magnitude of these failures tends to be exacerbated in settings that do not align with the underlying context of the data used for development, significantly impeding the ability to develop machine learning systems that perform well for underrepresented populations, generalize well across health systems, and are robust to changes in practice patterns over time. A key challenge is to develop procedures to anticipate and mitigate the consequences of these failure modes, both proactively during model development and continuously in deployed systems. ML4H 2021 will feature a thematic session and best thematic paper award on these challenges. Examples of topics within this area include:

  • Methods for learning models that are robust to distribution shift
  • Methods to improve or evaluate generalization across populations and health systems or under temporal non-stationarity
  • The design of models and training pipelines with inductive biases that promote stability
  • Procedures for auditing and monitoring machine learning models and systems
  • The design of stress tests to probe failure modes of machine learning models and systems
  • Case studies that characterize failure modes in the context of real-world datasets or deployed systems

Related works:

Dual Submission to NeurIPS Workshop

ML4H is introducing a new shared submission system program that allows other health-themed venues occurring around the same time as ML4H to accept submissions through ML4H rather than by building their own submission and review platform. This year we are piloting the program with the NeurIPS workshop,"Machine learning from ground truth: New medical imaging datasets for unsolved medical problems". Under this program, authors who submit to the ML4H extended abstract track on HotCRP will have the option to select whether they would like their paper to be considered for acceptance to the workshop in addition to ML4H. Please refer to this page for more details about the dual submission process.

Submission Instructions

Submissions (full papers and extended abstracts) are due on September 13th 11:59 PM AoE in the form of anonymized PDF files. There is no registration deadline. At the time of submission, authors will indicate whether they would like the submission to be in the proceedings track or the extended abstract track. As part of the submission, authors are required to fill out a submission form that will be visible to reviewers to help them assess the work.

All submissions for ML4H 2021 will be managed through the HotCRP system, accessible here. All submissions should be formatted using the ML4H 2021 LaTeX template.

ML4H 2021 template files

ML4H 2021 template on Overleaf

Note that all camera ready submissions must use this LaTeX template, and pending submission volume we may desk reject submissions that show a gross violation of formatting guidelines.

Author Response Period

Reviews will be released on October 11th. From October 11th to October 15th, 11:59 PM AoE, authors can submit a response to the initial reviews. Author responses may address any aspect of initial reviews, including by adding new results. However, author responses are limited to one page, including all figures, tables, and references, and should not include any links to external material. We reserve the right to solicit additional reviews after the author response period in the rare case that there are not sufficient high quality reviews to make a final decision.

Author responses must use the ML4H author response latex style files (to be released), must be fully blinded, and should be uploaded to HotCRP as a PDF file (author responses in the free-text box will be ignored). Author responses are due October 15th 11:59 PM AoE.

Reviewer Discussion Period

During the reviewer discussion period, reviewers and meta-reviewers are encouraged to discuss the paper, their reviews, and the author response. This process is aiming at seeking a consensus between reviewers and meta-reviewers. We ask reviewers to change their initially submitted review scores and recommendations during the discussion period, if applicable, and justify and state this in the discussion. Discussions will take place within HotCRP by using the comment function in a respective submission and should remain double-blind, i.e. comments may not deanonymize the authors or reviewers.

In general, these discussions will be between reviewers and meta-reviewers only. However, when further clarifications from the authors are necessary, reviewers may reach out to authors through HotCRP comments. It is only in response to such direct questions that authors should add comments beyond their author response, and said comments should be limited to directly answering the asked question. The reviewer discussion period formally ends on October 21 11:59 PM AoE , but discussions may be finalised earlier.

Registration Information

To promote community interaction, at least one presenting author of accepted works must register for the event. Registration details are forthcoming.

Contact Us

Please direct questions to: ml4h.2021@gmail.com and follow us on Twitter at @symposiumml4h.