MolSSI Workshop: Best Practices in Molecular Simulation
Organizers: Prof. Michael Shirts (University of Colorado Boulder) and Prof. David Mobley (University of California, Irvine)
Location: NIST Gaithersburg
MolSSI sponsored a workshop, August 24-25, 2017, focused around improving the way that molecular software is used, which can be equally important as improving the software itself. Molecular simulations could have an even larger impact than they currently do if we can better share the expertise it takes to generate sound results and validate them.
This workshop itself was intended to be part of a bigger initiative, and workshop participants were expected to be involved in a larger process of developing molecular simulation best practice documents and ‘checklists’. The workshop also addressed improving other approaches to educating starting researchers, including reviews and tutorial material. The scope of the workshop was classical molecular simulation, both molecular dynamics and Monte Carlo.
A report on the workshop’s outcomes is here.
Schedule
Thursday, August 24th
8:30 Shuttle from Hilton to NIST
9-10:30 The challenge:
- 9:00-9:15 Introduction (Michael Shirts)
- 9:15-9:40 Keynote “Making molecular simulation a reliable and reproducible tool” (Ed Maginn)
- 9:40-9:50 Discussion
- 9:50-10:15 Challenges in reproducibility and reliability simulation setup (John Chodera)
10:30-11:00 Break
11-12:30 The means to solve the problem:
- 11:00-11:15: LiveCoMS: A best practices overlay journal, i.e. How we are planning to get people credit for doing best practices documents
- 11:15-11:45: Discussion and brainstorming for the overlay journal
- 11:45-12:30: Discussion: How to best educate people beyond a journal: Tutorials, wikis, workshops, simulation requirements in other journals
12:30-1:45 Lunch
1:45-2:30 Discussion: How do we run/incentivize comparisons between codes?
2:30-3:30 Smaller group breakout session 1: discuss half of the working documents in parallel
3:30-4:00 Break
4:00-5:00 Smaller group breakout session 2: discuss other half of the working documents in parallel
5:00-5:30 Daily summary and wrap up
5:45 Shuttle to Hilton
6:30-8:00 Dinner (Rook’s Corner Restaurant, at the Hilton)
Friday, August 25th
8:30 Shuttle from Hilton to NIST
9-10:30 Discussion of best practices: Presentation of work so far
- (first half of topics) 10 min per presentation, 5 min discussion.
10:30-11:00 Break
11-12:30 Discussion of best practices: Presentation of work so far
- (second half of topics) 10 min per presentation, 5 min discussion.
12:30-1:30 Lunch (NIST)
1:30-2:30 Discussion of plans for moving forward. Topics include:
- Publicity for a overlay journal
- Recruiting more people for further documents
- Organization for a code comparison effort
- Other topics
2:30-3:00 Collaboratively writing outcomes document (document required by MolSSI)
3:00 Conference end
Attendees and Group assignments
| Basic simulation training (pre checklists) * | Jacob Monroe, Justin Gilmer, Heather Mayes, Avisek Das, David Mobley, Eliseo Marin-Rimoldi, Rommie Amaro, Julia Rice |
| Free energy calculations | John Chodera, Conor Parks, Heather Mayes, Antonia Mey |
| Best practices for simulation software/physical validation ** | Conor Parks, Pascal Merz, Justin Gilmer, Matthew Spellings, David Kofke, Bill Swope, Xiongwu Wu |
| Simulation setup | |
| Biomolecular | John Chodera, Antonia Mey, Elizabeth Ploetz, David Mobley, Rommie Amaro |
| Interfacial | Jacob Monroe, Andrew Summers, Kristin Ficthorn |
| Polymer | Coray Colina, Paul Saxe, Andrew Summers, Michael Fortunato, Sunny Hwang |
| Sampling assessment, convergence, and uncertainty estimation | Dan Zuckerman, Pascal Merz, Anthony Hazel, Andrew Schultz, Dan Siderius, Alan Grossfield, Daniel Roe, Paul Patrone |
| Property Calculation | |
| Transport properties in liquids/polymers | Ed Maginn, Richard Elliot, Sunny Hwang, Bill Swope, Daniel Roe |
| Fluctuation quantities | Elizabeth Ploetz, Ed Maginn, Baron Peters, Michael Shirts |
| Potentials of mean force | Anthony Hazel, Michael Shirts, Alan Grossfield, Baron Peters |
| Glass transition temperature | Paul Saxe, Richard Elliot, Michael Fortunato |
* This is a special category. It will likely eventually have more than just a best practices document, and also include tutorials as well (or at least links to tutorials!). We will assume that people looking at the other best practice documents can get the information they need here, which will require more coordinate between this one and others. This will almost certainly need to be divided into multiple subcategories, but will require coordinated work.
** This is also a special category. There’s a couple of different directions for this to go; a focus on checks that each code could run, which is sort of what I was thinking, but I want to give people a little freedom, since there was so much interest. We don’t want a focus on coding techniques – that’s a useful area, but I think we want to focus on the usage of and testing of code at this stage. Another potential area is comparison between programs; but I think this needs to be tackled in something a little bigger than just a best practices document, it might, however, be reasonable to develop a number of test cases that can be used.
Instructions for working groups for attendees
Our main goals up to and through the workshop:
- We have kickstarted 10-12 best practice documents that we can distribute, have people use, and iteratively improve
- It’s set up so that everyone gets lot of credit so it’s worth their while
Everything is oriented towards this. We expect it should just take a couple hours a week before the conference to get the process rolling.
We have identified 12 initial topics based on interests expressed in registration, listed below. We have put at least three people on each topic, and no person is in more than two groups. If you would prefer to be in different groups, or would like to be in more, or have any other questions, let me know! It’s perfectly acceptable to switch between groups, or modify the topic area slightly.
Working with your groups, we ask that you:
By Friday, July 28th:
- Brainstorm the resources and information needed to generate a best practices document for the subject. Specifically:
- Identify a point person in each group that you can trust to make sure the tasks above are accomplished and indicate this person to us.
- Identify 1-4 additional people who really should be involved to make the best practices document. Decide whether to invite them at this point or to wait until after the conference.
- Identify the papers in the field that the best practices will be drawn from.
- Identify if there are new results that need to be generated to nail down certain best practice issues.
- Generate an initial bullet-point list of best practices
- Identify all the main points that need to be considered when performing the simulations/calculations identified below
- Doesn’t yet need to be more than 2-4 pages
- Keep code agnostic for now.
- Assume, for now, general understanding of molecular simulation knowledge.
- If you decide prerequisite knowledge is expected, pass that information to the group working on basic simulation training.
- Decide if the topic really should be split into more than one document (and decide on which of those documents to focus on).
- Google Doc would probably be the best way to collaboratively edit them at this point. I’ve created as shared Google Drive folder for these (link sent to participants).
- Identify all the main points that need to be considered when performing the simulations/calculations identified below
Between July 28th and August 24th, 9:00 a.m. (start of workshop)
- More information emailed directly to the participants.