peter reinmann2.jpeg

Convenor: Prof. Peter Reimann, CRLI

Communication and collaboration competences are mentioned prominently amongst the graduate attributes of the University of Sydney, they are also listed as essential skills on job descriptions world-wide. However, how to systematically develop such competences remains largely unspecified, and how to assess them is a matter of wide-spread and contentious discussion.

In this workshop, instead of providing ready-made solutions that can be too generic to be directly applied to discipline-specific teaching, we aim to collectively develop a number of educational design patterns. These patterns will provide answers to two guiding questions: (i) how to develop collaboration competences, and (ii) how to assess them for formative purposes -providing students with feedback. As first suggested in architecture (Alexander, 1979), a pattern describes an effective solution to a recurrent problem embedded in a specific context. In education, where solutions take the form of learning designs, we speak of pedagogical design patterns and of assessment design patterns.

We are proposing design patterns as an appropriate means not only for documenting current teaching and assessment practices, but also as the basis for initiating and sustaining a process of continuous improvement and innovation. Patterns are descriptive and they have the potential to guide knowledgeable action. In order to achieve these aims they need to be developed in a participatory manner involving all stakeholders, so as to capture multiple areas of expertise that can lead to the development of new practices. To this end, after the workshop, we will provide an online platform where the patterns we develop will be made available for further refinement.

Teaching staff—with an interest in developing graduate attributes in the areas of collaboration, communication and leadership—will benefit from participating in this workshop.

Event details
When: Friday 19th May 2017, 1-5pm
Where: Room 221, The Educational Design Research Studio (EDRS), Education Building (A35).
Please refer to map for further directions.


The increasing availability of data about the interactions occurring in a learning experience through technology mediation offers the opportunity to explore new ways to support students. However, having a comprehensive set of data points is far removed from effective support actions. Learning is influenced by a large variety of factors and variables. Research areas such as learning analytics need to take into account both theoretical and contextual factors to achieve improvements on a learning experience. In this talk we will explore how this connection can be established and discuss current research initiatives in this space.


Linked (Open) Data is a technology and a political initiative designed to facilitate general access to high quality data from diverse domains. It is a movement which inherits technologies from the Semantic Web effort, and to some extent grew out of some frustration with that initiative. I will discuss some of the background and the current state of the art in the linked data initiative. I will describe DBpedia, an automatically extracted linked dataset from WIkipedia. Finally I will demonstrate an iOS application I developed for using linked data in an educational tool, which has highlighted some of the frustrations with the current status of linked data.


Csaba Veres
I am an Associate Professor from the University of Bergen at the Department of Information and Media Studies, and associated with the Centre for the Science of Learning & Technology (SLATE) at UiB.

I received my Ph.D. in Psycholinguistics, where I was interested in the way semantic representations were computed in sentence comprehension. But my interests were more broad, encompassing everything to do with conceptual structure and how it is learned, and especially the way this interacted with language. In the years following my Ph.D., I drifted away from Psychology and into Computer Science, where I have maintained my interest in semantics and language through my work with semantic web technologies. I prefer working on practical rather than theoretical issues, which is one reason I drifted away from theoretical, experimental cognitive psychology. My focus has been to discover ways in which linguistic knowledge can be used in semantic web applications. This has given birth to a number of applications including semantic text markup and tools which support full blown ontology construction through natural language. I have also been working on what we call Social Semantic Information Systems, which aim to bring semantic technologies into social web applications.

You can also find Csaba on his website and Twitter.


The Sydney Research Excellence Initiative (SREI) — SREI 2020 is a new program to help Sydney researchers test new ideas, push disciplinary boundaries and identify ways to scale up their research.

Closing date for applications is Friday 5.00pm 21 April

This Scholarship (stipend) is offered to current Higher Degree Research (HDR) candidates from the University of Sydney to support their participation in the SREI 2020 project "Understanding and facilitating learning in emerging knowledge co-creation spaces", within the Centre for Research on Learning and Innovation (CRLI). Recipients will work as a part of a multidisciplinary team implementing one of the following six seed projects:

  1. Knowledge co-creation in health and medical technology innovations
  2. Boundary crossing between learning technology research and the educational technology industry
  3. Knowledge co-creation in school-university partnerships
  4. Knowledge practices in learning analytics
  5. Learning for the workplace through innovation and knowledge co-creation
  6. Research dissemination via knowledge co-creation



Reflective Writing Analytics can be thought of as simultaneously working across two epistemic domains: the psychosocial domain that encompasses aspects like personal disposition, cognition and the pedagogical context; and computational domain which includes machine representation, analysis and feedback of analytics to the user. How do we work effectively across these two domains for the benefit of learning and teaching? This is a core question underpinning the research we undertake at the Connected Intelligence Centre (CIC), University of Technology Sydney (UTS).

In this first half of this session, I will present some of the ways that we approach Reflective Writing Analytics at CIC. I will highlight some key characteristics of the psychosocial and computational epistemic domains and outline some of the challenges in bringing them together. To illustrate these ideas, I will draw on examples from our recent work in analysing reflective writing for the purpose of providing actionable feedback to students.

In the second half of the session there will be the opportunity to discuss as a group how similar approaches to those used at CIC might be currently, or in the future, assist the work of those present. By sharing ideas, and raising questions it is hoped that we can harness the collective intelligence of the session, sparking ideas that are helpful for your own work.


andrew martin.jpeg

The research in this presentation reports on real-time longitudinal intra-individual data collected in mathematics and English lessons, every school day, across four school weeks. A total of 113 boys and girls in Year 7 from two Australian schools participated.

Using mobile technology (e.g., smart phones, laptops, tablets) to capture intra-individual real-time data, a four-level model was explored, consisting of between-lesson (within-day) ratings at the first level (up to 2 lessons per day), between-day ratings at the second level (5 days per week), between-week ratings at the third level (4 weeks), and between-student ratings at the fourth level (thus, 40 possible time points per student). Multilevel modeling showed substantial between-lesson (within-day) variability in motivation and engagement (M = 34%) and substantial between-student variability (M = 62%). There was not so much variability between days (M = 2%) or between weeks (M = 2%).

We propose the study offers insights for motivation and engagement theorizing (particularly around stability and developmental issues) and technological and logistic guidance for collecting real-time data. Furthermore, these findings derived from boys and girls in two schools replicate those from a previous study (also discussed in this presentation) conducted among a small sample of boys.

The findings again show that every minute of every day for every student matters. To the extent that this is the case, there are policy implications for daily school timetabling, teacher training to better support motivation and engagement through the school day, and the use of mobile technology to monitor students and enable responsive pedagogy and intervention in real-time.



kate thompson.jpg

The current excitement in schools around Makerspaces, robotics, and STEAM education is underpinned by substantial research in how students can learn about Science, Technology, Engineering, the Arts, and Mathematics, in a connected, collaborative, interdisciplinary approach. The investment in innovative learning spaces, and resources such as 3D printers and electronics provides us with an opportunity to change the way we think about teaching and learning in STEAM disciplines. 

Dr Kate Thompson is the head of the Creative Practice Lab (CPL) at Griffith University. The CPL combines teacher education, digital technologies (including robotics and digital fabrication), with state-of-the-art video recording and online collaboration systems (including a newly designed Virtual Internship). The CPL aims to provide opportunities to engage with innovative research, and to engage in research partnerships with schools to understand pedagogical approaches to STEAM in face-to-face and online environments.

Kate will present stories from her research, including the OLT-funded STEP-UP project, connecting school students, STEAM experts, and in-service and pre-service teachers. This work allows us to understand those moments in which students connect their knowledge and skills across the STEAM fields to answer questions, solve problems and create products. 

Kate will discuss the implications for teacher practice and preservice teacher education.



Professor Peter Reimann

The widespread availability of learner-related data has the potential to empower students, teachers, parents and school leaders by providing critical insights into the learning process. However, fostering a culture of data-informed learning and teaching in schools remains a significant challenge. This is in part due to capacity: Teachers are by and large not prepared for advanced data practices, and teacher education providers are currently not well prepared to develop that capacity. Research on developing data literacy for teachers, research so far has focussed mainly on three aspects: 1) analysis of the components that make up this literacy, 2) analysis of existing teacher education programs, and 3) studies on professional development programs conducted with in-service teachers. After an overview of the state of the art, I will address questions regarding What to learn and How to learn about data literacy in more detail, focusing on pre-service teacher education because very little is known at this stage about how to design for and support the learning of education students.

Event details
When: Wed April 12, 2017, 11.30am– 1.00pm (this is a brown bag seminar, attendees are welcome to bring their lunch)
Where: Room 612, Education Building A35
This seminar will not be available online or recorded.
No need to RSVP, just come on the day.


What are the benefits of providing peer feedback (online) and how can it be made even stronger?

Questions of effectiveness and quality of peer feedback and peer assessment have been very actively discussed from the perspective of the receiver of feedback, however the benefits for the provider of feedback messages have received much less attention, at least in Higher Education. But the benefits could be substantial: The process of peer feedback engages students actively in learning, helps develop self-management and judgment, strengthens the capacity for self-assessment, helps develop subject knowledge, enables students to receive feedback faster and promotes social interaction.

Understanding the benefits to the student providing the feedback is becoming more important as the opportunities for engaging in peer tutoring and peer assessment practices explode in the online space. In addition to designed peer learning practices we need to consider informal peer help and peer tutoring episodes.

Peter will give an overview of recent research on the online provision of peer feedback in relation to research on peer feedback and peer tutoring more generally. He will speculate on how learning benefits for the student provider of online feedback might be increased, based on explanatory models for the learning from teaching effect. He’ll end with some thoughts on how technology can help, not only with providing peer feedback, but with the learning that arises from providing feedback.

Prof. Peter Reimann is the co-Director of the Centre for Research on Learning and Innovation (CRLI) at the University of Sydney, (formerly the CoCo Research Centre), in addition to having continued involvement in European Commission funded projects in IT research and development for learning. His primary research areas include cognitive learning with a focus on educational computing and the development of evaluation and assessment methods for the effectiveness of computer-based technologies. Current research includes the analysis of individual and group problem solving/learning processes and possible support by means of ICT, and analysis of the use of mobile IT in informal learning settings (outdoors, in museums, etc.).

Event details:

7 April 2017
12 – 1pm
Room 218, Level 2 South,
Fisher Library


At CRLI, we run a weekly seminar series hosting local and international experts who present research on learning and educational innovation in an informal setting. Seminars run on most Wednesdays in semester.

Upcoming seminars in 2017:
A glance of this year's seminar
12-Apr-2017—Peter Reimann
26-Apr-2017—Andrew Martin
3-May-2017—Kate Thompson
17-May-2017—Andrew Gibson
24-May-2017—Deborah Richards
31-May-2017—Kathryn Bartimote-Aufflick
7-Jun-2017—Learning Analytics Research Group (LARG)
14-Jun-2017—Rachel Wilson
28-Jun-2017—Maria Souza e Silva
2-Aug-2017—Louise Sutherland
9-Aug-2017—Nick Hopwood
20-Sep-2017—Tom Carey
4-Oct-2017—James Dalziel
1-Nov-2017—Learning Analytics Research Group

Please note we are in room 612 of the Education Building. We are always looking for more speakers, topics and ideas. If you would like to suggest a seminar topic, propose a speaker (including yourself) or provide feedback, we would love to hear from you at

When: Wednesdays, 11.30am. Seminars usually run for an hour followed by a 30min Q&A session.
Where: Rm 612 of the Education bldg. (Unless otherwise specified in the seminar's description page).
Brown bag: You are welcome to bring your lunch to these events.

Are you a visual learner? No? Perhaps you are an auditory or a kinaesthetic learner? The idea that people differ in what modality they learn best in, and that knowing this should influence how one is taught is known as "learning styles". This idea is one of the more enduring "neuromyths" in education. Earlier this week, thirty leading researchers in neuroscience, psychology, and education signed a letter to the Guardian strongly condemning learning styles approaches—"No evidence to back the idea of learning styles".

The idea of "learning styles", however, is not the only neuromyth that makes its way around our education system. Paul Howard-Jones reviewed the broader issue of educational neuromyths in the prestigous neuroscience journal Nature Reviews Neuroscience in 2014. One of the findings discussed was that of a survey of practising teachers in five different international contexts. These teachers were asked about various neuromyths and whether they believed them to be true. The table below is taken from this paper, and shows the results of this survey.

Screen Shot 2017-03-15 at 12.06.13 pm.png

And while on the subject of the brain, we were asked a question about how our brain decides which words to use. For example, as I type this now, why am I choosing to use these words, rather than different ones?

First of all, we still have much to learn about the brain, this issue included. However, one generally accepted idea is that our brains work to constantly gather evidence in order to better understand and make predictions about the world our bodies inhabit. This, as various arguments go, is what our brains evolved to do. So, much of our unconscious decision making about which words we use is governed by our brain picking up subtle contextual cues in the environment that bias it toward certain decisions.

Mathematically, neuroscientists sometimes use what is called a drift-diffusion model to analyse this phenomenon and how it might correlate with certain brain signals.

drift model.jpg
A drift-diffusion model. The horizontal axis represents evidence gathered over time and the vertical axis represents nearness to a decision boundary (Zhang & Rowe, 2014)

To make this more concrete, I thought up an analogy on my walk into CRLI this morning. Imagine the situation of two people walking along a footpath towards each other. When deciding who walks on what side, in Australia, we have a leftward bias influenced by our traffic laws. But what happens if these people are already quite close and just happen to be walking on their respective right-hand sides? Our brains, here, recognise it is easier to stick on the side we are already on, and this overrides our natural tendency to veer left. In this instance, our brains are gathering spatial and movement information from what we see and hear to decide where we walk, and this is also influenced by cultural context. Back to the word choice example, I might normally write one word on a blog (cultural and experiential biases) but I might subconsciously choose another one influenced by what I might have been reading that day. At a higher level, it is similar to how we have ideas. For example, my walk this morning influenced the ideas I talked about in this blog.


Howard-Jones, P. A. (2014). Neuroscience and education: Myths and messages. Nature Reviews. Neuroscience, 15(12), 817-824.

Zhang, J., & Rowe, J. (2014). Dissociable mechanisms of speed-accuracy tradeoff during visual perceptual learning are revealed by a hierarchical drift-diffusion model. Frontiers in Neuroscience, 8(8), 69.

In the last couple decades, one of the most talked about ideas in education has been that of the "flipped classroom"—see for example In a traditional classroom, we receive a lecture on a topic, then get practice in applying the topic for homework. A flipped classroom "flips" this structure. In other words, we watch a video of a lecture at home and then practice and apply the concept in the classroom. The advantage of a video lecture is that one can pause or rewatch sections of a video if one fails to understand, all without time pressure or peer pressure (e.g. not being confident enough to ask a question). The advantage of 'homework' in the classroom is that the teacher can more actively support the students during formative stages of applying a concept and intervene more immediately when needed.

Despite a growing interest in flipped classroom models of learning, there has been very little research that has empirically evaluated its efficacy. And despite its prominence in the popular consciousness, there are a number of educational researchers not so convinced of its claims to "revolutionise education". So given this, an international team, including CRLI core member Abelardo Pardo, has taken a critical look at this issue in a new paper in the journal The Internet and Higher Education. Working with methodologies from the field of learning analytics, they identify 4 different learning strategies and 5 different student profiles that tend to occur within flipped classroom teaching. Having a finer grain of analysis, as offered here, enables educational researchers and designers to better understand and improve how we might use flipped classroom teaching.

See the original paper for details.

Screen Shot 2017-03-01 at 4.36.18 pm.png
Two representations of student activity patterns for A) students performing above the 90th percentile, and B) students performing below the 25th percentile

And as a bonus piece of good news, Abelardo has just been appointed a "Senior Fellow" of The Higher Education Academy: a professional institution promoting excellence in higher education. The Senior Fellow distinction is given to individuals that demonstrate a thorough understanding of effective approaches to teaching and learning support as a key contribution to high quality student learning.

Design image.jpg

Normally, we tend to think of teaching just as something that teachers do, and perhaps if the teacher is diligent, the teacher might engage in some lesson planning to prepare their teaching. We tend to not, however, to think of teaching as something that can be designed. Educational/instructional design is something that is only beginning to be explored more seriously, and our own team here at CRLI has a few things to say about the matter.

A new paper in journal Computers in Human Behavior does just this. A multi-institutional team, including our centre co-director Peter Goodyear, some former CRLI/CoCo academics, and other collaborators, explores the latest research coming from our innovative Educational Design Studio (EDS). The results speak to the powerful affordances of both digital and analogue tools in a design environment, and the importance of collaboration in the design process.

Check out the paper for more details

Beijing Normal University circle logo.jpg

One of our associated research groups, the Learning, Cognition, and Brain Sciences research group (LCBS), will be hosting a couple guests from Beijing Normal University the coming week. Dr. Qinhua Zheng and Dr. Jingjing Zhang will join the LCBS meeting and informally discuss their research, primarily involving learning analytics, educational data mining, and educational technologies more broadly. Anyone interested in joining the meeting and hearing about their research is most welcome to do so.

Date: 14th of February; Time: 10am-12pm; Location: CoCo Lab (Room 237 in the Education Building, University of Sydney)

Dr. Qinhua Zheng is associate dean of School of Education Technology and director of Research Centre of Distance Education at Beijing Normal University (BNU). Since 2009, he has worked as an associate professor in Research Center of Distance Education, BNU. Areas of his research interests include the cost-effectiveness analysis of distance education, quality assurance of e-learning,Massive Online Open Courses; learning analytics. These years, his researches mainly focused on how to build the model to explain the online learners’ competency during their web-based learning, and how to use data to show and assess the learners’ situation. He has developed the software “Wisdom Line” to give supports to the online learning institutes in China based the researches.

Dr. Jingjing Zhang received her BSc in Computer Science from BNU, an MRes from University College London (UCL), an MSc and a DPhil from the University of Oxford. As an undergraduate, she was awarded 2003 AIEJ Scholarship for a one-year exchange study at Tokyo Gakugei University. At the University of Oxford (MSc, DPhil), she was a Clarendon scholar and a member of Brasenose College (funded by ORS scholarship). She now directs the Big Data Centre for Technology-mediated Education at the Faculty of Education of BNU, specialising in learning and technology. Before joining BNU, she was first trained in Directorate for Education, OECD Paris, and then interned at the Department of Management, the UN headquarters New York. Dr. Zhang’s work focuses on developing data mining techniques (e.g., complex networking analysis) to explore human relationships and activities online, particularly in the learning sciences. This includes the impact on learning and collaboration in using open educational resources (OERs), massive open online courses (MOOCs), and knowledge visualization.

Screen Shot 2017-02-07 at 11.10.30 am.png

We bring good news!

CRLI researchers have won funding for an ambitious, interdisciplinary project to study how people learn to co-create knowledge. Funded by the Sydney Research Excellence Initiative (SREI 2020), researchers from CRLI and across the university will come together to look at how people learn in complex, interdisciplinary environments from schools, to universities, to workplaces and cutting-edge industries. More than simply learning old facts, however, the project will focus on understanding how we learn to co-create new knowledge with others. Increasingly it is realised that this ability to work with knowledge, and to do so in varied interdisciplinary teams, is increasingly important for life and work in a knowledge-generating society. An understanding of how we learn to do this, therefore, is important!

So stay tuned for updates as this project develops. And congratulations to the team, led by Associate Professor Lina Markauskaite. Exciting times ahead for CRLI!

Instructional wordle.jpg

The job of ranking the academic productivity of Universities is always a difficult one, and the results of which are invariably contested and fought over. Despite this, a team from the US has recently offered a ranking in the area of ‘Instructional Design and Technology’ that looks at a number of metrics and offers some thoughts on the field at large (West, Thomas, Bodily, Wright & Borup, 2016). One of their key metrics used to determine the outcome was to look at a selection of 20 of the discipline’s key journals, and to see which university produced the most papers in these journals. Looking at a period between 2005-2014, and a total of 1496 individual papers, our very own University of Sydney came a quite respectable 12th place.

While not quite a podium finish, we take the gold medal amongst Australian universities and place above a number of prestigious American universities. Not bad for little ol’ Sydney (the rough quantitative approach taken here favours larger universities).

West, R. E., Thomas, R. A., Bodily, R., Wright, C., & Borup, J. (2016). An analysis of instructional design and technology departments. Educational Technology Research and Development, 1-20.


Ryan Baker is visiting on his way to a keynote at the ASCILITE Conference in Adelaide. He will be giving a talk at 3pm on Friday 25 November at the University of Sydney, all welcome! Friday 25 November, 3pm, rooms 249 & 250, Level 2 South, Fisher Library (F03), University of Sydney. Campus map:

Ryan Baker.jpeg

Modeling Complex Skill with Educational Data Mining

Abstract: In recent years, the emerging methods of educational data mining have made it possible to model performance on complex skills occurring within online education, making it possible in turn to measure learning of these skills over time. In this talk, I will discuss my lab’s work to model the development of these skills, both in structured contexts such intelligent tutoring systems, and less structured contexts such as simulations and serious games. For example, I will discuss our work to infer when students are able to successfully design controlled experiments within simulations, and when they are able to successfully navigate a complex virtual environment to gather evidence that enables them to answer causal questions about events in that virtual environment. By modeling complex student skill and inferring students’ knowledge of these skills, we make it possible to create educational and training environments that can support students in developing robust and useful competencies.

Ryan is Associate Professor of Cognitive Studies at Teachers College, Columbia University, USA. He is in Australia as a keynote speaker at the 2016 ASCILITE Conference, and we’re fortunate to be able to have him with us in Sydney while he’s visiting.

LARG.jpgThe Sydney Learning Analytics Research Group (LARG) is excited to offer a conference travel grant of $3,500 to attend the 2017 Learning Analytics & Knowledge (LAK) Conference to be held in Vancouver, BC, Canada 13-17 March.

The call for applications for this grant is now open, and the due date is Monday 16 January 2017. Applicants must have a submission (of any type) accepted for presentation at LAK 2017, and be either a current staff member or current student of the University of Sydney. Submissions for LAK are currently open - there are several deadlines, the last of which is 2 December 2016.

More information is available, including how to apply and the conditions of the grant at

Join us on November 2nd for our last seminar of 2017, the "Learning Analytics Research Group (LARG) showcase session", with presenters Jess McBroom and Associate Professor Kalina Yacef.

LARG is a joint venture of the newly established Quality and Analytics Group within the University's Education Portfolio, and CRLI. The key purposes for establishing LARG are to:
:: build capacity in learning analytics, for the bevefit of the instituion, its staff and its students
:: generate interest and expertise in learning analytics at the University, and build a new network of research colleagues
:: build a profile for The University of Sydney as a national and international leader in learning analytics.

This seminar is the first event in which the LARG will showcase two of ts recent projects.
Presenter: Jess McBroom
Presenter: Associate Professor Kalina Yacef

Event details
• When: 11.30am to 1.00pm on 2 Nov 2016. This is a brown bag event, you are welcome to bring your lunch to eat.
• Where: Room 612 Education Building A35
• This seminar will not be available online or recorded.

The CRLI Wednesday seminars (formerly CoCo and STL) run on most Wednesdays in semester and host local and international experts who present research on learning and educational innovation in an informal setting.

Join us on October 19th for "Sites of Epistemic Cognition", a CRLI seminar with Dr Simon Knight, Research Fellow in Writing Analytics at the University of Technology Sydney.

Simon-Knight267.jpgSimon's PhD research investigated epistemic cognition – cognition regarding the source, justification, complexity, and stability of knowledge – in collaborative information tasks. Students worked on separate computers making use of a tool (Coagmento) that facilitated their activity, providing a chat and collaborative text editor, and tracked their activity.

In this talk, I’ll discuss developing work on conceptualising the design of that research in terms of ‘sites of epistemic cognition’: situations; activities; products; and actors. In a parallel line of work, I have been re-conceiving ‘epistemic cognition’ in light of recent moves in the philosophical literature on ‘social epistemology’. The talk will introduce this novel account, illustrating its use as a lens onto epistemic dialogue.
Event details
• When: 11.30am to 1.00pm on 19 October 2016. This is a brown bag event, you are welcome to bring your lunch to eat.
• Where: Room 612 Education Building A35
• This seminar will not be available online or recorded.

The CRLI Wednesday seminars (formerly CoCo and STL) run on most Wednesdays in semester and host local and international experts who present research on learning and educational innovation in an informal setting.

About Us

The Centre for Research on Learning and Innovation (CRLI) aims to provide a focus for the university’s research on learning and innovation. Formed from CoCo and the STL research network, we have strong roots in Education, with substantial involvement from Engineering & IT, Science, Health Sciences and Medicine.

About the Blog

Research by the University's Centre for Research on Learning and Innovation (CRLI).