UBC faculty and students gathered at the Open Scholarship in Practice seminar, held in the Irving K. Barber Learning Centre’s Lillooet Room on September 29, to learn more about nascent technologies and innovative teaching approaches that encourage open scholarship.
Advocates of open scholarship spent the day engaging the audience about what works – and what doesn’t – when scholars try to bring the scientific process closer to the surface. The seminar’s speakers emphasized that open scholarship has wide-ranging applications across different fields of study, and that its collaborative structure actively encourages interdisciplinary work. Open scholarship practices can also boost academic accountability and rigor, since more people can read, review, and replicate studies that are readily available.
Open Scholarship, Transparency, And Reproducibility
The seminar began with a welcome from Susan Parker, UBC’s new University Librarian. Parker introduced Vice-Provost and Associate Vice-President Academic Affairs Dr. Eric Eich, who took the podium to talk about an issue plaguing the scientific community, at UBC and elsewhere: a lack of transparency and reproducibility.
One of the most effective ways for scientists to verify others’ results is to reproduce the study in question, or redo it under the same conditions. However, when scientists perform studies but do not share, or selectively share, the data used to obtain their results, it is harder for others to replicate those studies. This lack of complete data sets, combined with a funding model that heavily incentivizes new research questions over reproductions of existing studies, means that scientists end up doing very few replicative or follow-up studies at all.
Dr. Eich estimates that of all “the data published out there, all the papers, that are direct replications, it’s anywhere from one-tenth of one percent to one-thousandth of one percent,” Dr. Eich said. “So it tends to be vanishingly small.”
He also pointed out how sequestered data often hides poor scientific practices. Open practices can reduce the incidence of “p-hacking,” or analyzing trends in data sets without a prior hypothesis in mind. P-hacking, in turn, usually leads to “HARKing” (or Hypothesizing After the Results are Known) in the interpretive stage of the scientific process. Such shortcuts flip the scientific process backwards and turn out results that might be correlative, but not necessarily causal.
To combat these trends, Dr. Eich, along with Dr. Jason Pither, Associate Professor of biology at UBC’s Okanagan campus, formulated a strategic initiative proposal in March of this year for advancing uniform transparent science policies at the university level. Broadly speaking, the initiative will promote a detailed orientation to open science options and best practices. Dr. Eich suggested UBC “should produce graduates for whom best practices in open science are as routine as donning a safety coat and glasses within a lab.”
“We’re all sitting here talking about this as a crisis. Hopefully, there will be no crisis in the future, once people start practicing in a more responsible manner,” he added.
Undergraduate Students: Producers And Beneficiaries In Open Scholarship
Not only do researchers benefit from open scholarship – it’s a boon to undergrads too. UBC Alma Mater Society (AMS) Campaigns and Outreach Commissioner Christina Ilnitchi, Professor of Teaching Dr. Christina Hendricks, and Centre for Teaching, Learning and Technology (CTLT) Open Education Initiatives Strategist Will Engle spoke about the ways UBC is reaching out to nearly 52,000 undergraduate students between the two campuses.
Ilnitchi spoke about the AMS’s textbook savings program, #textbookbrokebc. The initiative promotes open education resources (OERs) to help supplant pricey course textbooks and ease undergraduates’ growing financial burdens.
To underscore this cost crisis, AMS stood outside the Bookstore during the first week of class, asking students if they would volunteer to write down how much they had just spent on books.
“We had over 1,000 students participate, and they spent, collectively, around $400,000,” Ilnitchi said.
That’s certainly cause to start considering alternatives. But what are these open education resources? Are they any good? Do many classes offer them?
A handful of classes, including some prerequisites like Physics 100 and Chemistry 211, now use open, free textbooks for their readings. A few more take this one step further, encouraging students in the course to help contribute to an open textbook or other editable course materials, like Wikis. While the OER initiative is off to a promising start, the Alma Mater Society realizes that it is imperative to get more faculty on board. The AMS will soon embark on a “listening tour” to get instructor input on current barriers to OER adoption and how to overcome them.
Engle and Dr. Hendricks picked up where Ilnitchi left off, detailing the pedagogical strategies that make UBC’s open education resources possible. The “student as producer” model, in which undergraduate students internalize concepts by actively working together to generate and share content, helps them exert greater control over their learning process.
Open approaches also change the workload dynamic of courses that adopt them in innovative ways. To illustrate, Dr. Hendricks drew a distinction between “disposable assignments” and “renewable assignments.” Disposable assignments are given to the professor, graded, commented on, handed back, and are probably eventually thrown away.
“But to only do that is missing out on other interesting things students could be doing,” Dr. Hendricks said. “So we also suggest we think about renewable assignments. Renewable assignments are things that add value to the world in some other way besides just getting a grade…. and it’s renewable in the sense that somebody can take that thing you’ve produced and add to it, edit it, delete some aspects, and use it in a different way.”
At UBC, undergraduate students who take OER-based courses have helped develop front-facing web content. Dr. Hendricks oversees the Open Case Studies site, which features student-written overviews of issues in environmental sustainability, social justice, and digital life.
UBC students have also contributed to existing Wikipedia pages, or built entirely new ones from the ground up. Engle pointed out that students’ Wikipedia contributions are subject to a version of peer review when they are challenged in article Talk pages. This exercise, Engle said, gives students the academic confidence to continue contributing to the discourse in future.
And when big news happens, there’s an off-chance that UBC student-written Wiki content can suddenly reach a very broad audience.
“The Gabriel García Márquez article in Wikipedia was originally largely written by UBC students,” Engle said. “When Gabriel García Márquez died, the page that the UBC students wrote had been seen 200,000 times in one day.”
Writing (And Revising… And Re-Revising) Open Textbooks
With all of the excitement surrounding open scholarship, it is sometimes easy to forget just how much time and effort it has taken to get where we are now. Dr. Leah Edelstein-Keshet, Professor of mathematics at UBC, talked about the long-running “grassroots effort” of implementing open texts within her department.
Nearly 20 years ago, Dr. Keshet, along with Drs. David Austin and William Casselman, began to share scanned and HTML-based lecture notes and supplemental materials on the departmental website. By the early-mid 2000s, Drs. Austin and Casselman were working together to expand the department’s online offerings to math labs, applets, and spreadsheets.
For the past several years, Dr. Keshet and her graduate assistants followed in the footsteps of these early open practices developing a full, freely-available online textbook (and an accompanying Wiki) for Math 102, an applied “life-sciences calculus class.” Each of the 900 or so UBC students that take Math 102 now use the open text, which has also been adapted for Simon Fraser University’s introductory calculus courses.
That said, Dr. Keshet warned the process of open textbook writing can be long, labour-intensive, and not for everyone. Between her own time and seven volunteer graduate students, she noted it took thousands of work-hours – “fueled by free sushi and pizza” – to get the Math 102 text to where it is now.
All of the graduate students that worked closely on the project have since finished their degree programs and left the university, making the textbook harder to maintain.
To keep the text fresh, Dr. Keshet and Will Engle now plan to copy the existing data over to a sandbox forum where undergraduates can contribute. “We’ll start out with a pilot project this semester and see how this works,” Dr. Keshet said.
Data Science In The Open Cloud: Syzygy.Ca
UBC mathematics professor Dr. James Colliander and UBC mathematics instructor Dr. Patrick Walls demonstrated the capabilities of syzygy.ca, which hosts Jupyter notebooks in the cloud.
Jupyter notebooks are powerful data analysis tools that use kernels to adapt to many different programming languages. Syzygy builds a flexible, lightweight infrastructure around Jupyter, making it easier for scientists and scholars to instantly share their data sets and code.
With Syzygy, UBC users don’t have to download any software to their machines; they simply go to ubc.syzygy.ca, sign in with their Campus-Wide login, and they’re ready to start working. 4,000 people from 15 universities have used Syzygy so far.
Dr. Walls teaches two UBC courses, plus a seminar series, that all use it extensively.
“It has changed the way I teach in the classroom, for sure, because it’s not about computers in the computer lab, using proprietary software, where you can only use it for one hour per week,” he said. “It’s online, it’s free to students, it’s free to faculty. You can do computing anywhere, anytime.”
Introducing Open Science Framework
Concluding the event, Jennifer Freeman Smith of the Center for Open Science delivered a half-day training workshop on its flagship Open Science Framework platform, repeating a session held at UBC Okanagan the previous day.
Reiterating many of the day’s themes, Freeman Smith took a holistic view of scientific discovery, noting “the published report is a pretty poor archive of what’s been done.” She demonstrated how, using OSF, researchers can manage all stages of their projects and collaborations, including pre-registration of studies.
OSF is also a powerful tool for sharing and managing data. Suggesting how using OSF functionality and tracking capabilities can help researchers, she concluded: “Your most likely collaborator is you, six months from now.”
The event organizers thank all those who presented and attended this information-packed day. Watch for more open science workshops and events at UBC in the coming months.