Using Gradescope for transforming easily your paper-based homework into online assignments

Helmut Neukirchen, 15. March 2020

Update: now that everyone has Canvas, you can easily sync Canvas and Gradescope with respect to student roster and assignments:
when creating an assignment in Canvas, select as assignment type/tegund skill the type external/ytra and then type in "Gradescope" and let Canvas search for Gradescope and select then "Gradescope" from the results list.

See how to sync Canvas and Gradescope.
(And ignore those parts below that only apply when you do not sync with Canvas...)

The Computer Science department of the Iðnaðarverkfræði-, vélaverkfræði- og tölvunarfræðideild (IVT) of Verkfræði- og náttúruvísindasvið (VON) uses already since 2015 Gradescope for an easy online assignment submission by students and for extremely convenient and time-saving online grading and feedback by teachers.

Both students and teachers love it (e.g. in the 2019 IVT self-evaluation, the students explicitly requested more usage of Gradescope and independent from that the teachers requested IVT deild to spend money to get a full license). IVT deild has paid for a full-feature institutional license that everyone who registers with hi.is email can use for free in 2020. (Update: due to COVID-2019, Gradescape just made all features anyway available for free.)

The approach of Gradescope is that you can keep your traditional approach (so changing to Gradescope is really easy):

  • For assignments/homework during the semester, students upload on their own their solutions: either taking a photo of their paper solution or (as anyway most students typeset their solution electronically) upload a PDF of their solution, and mark on their own where on the uploaded pages the solution can be found.
  • Final exams are done as usual on paper, but the teacher defines boxes where to fill in the answers so that Gradescope knows where to look for the answers and the teacher scans later in and uploads the exam solutions to be able to use the convenient online grading features.
  • In addition, pure online assignments/exams are supported, i.e. instead of uploading a solution, students answer some web form.

Image copied from https://www.gradescope.com/

If you have any questions, you are welcome to contact Helmut Neukirchen. But first have a look at the info below:

Demo videos (each 2-3 minutes):

Teacher creating assignment

Student submitting assignment

Teacher grading submission

Artificial Intelligence (AI)-based image recognition to group solutions that look similar and should thus all get the same grading

E.g, in a course with 40 students, grade the 20 completely correct assignments with one click, the 10 assignments that make all the same mistake with one click, and the 5 empty assignments with one click, so that only the remaining 5 assignments need your attention.

Gradescope is easy

As you see, this is all very easy and a natural (but faster) extension of your paper-based assignment workflow.

(Note that my experience is based on Computer Science assignments and exams, where answers typically fit on one page, but grading is even fun for programming assignments where source code submissions are long. But if you scroll one page down on https://www.gradescope.com, you see also Gradescope being applied to other disciplines than Computer Science.)

As long as not every course uses Canvas, students need to be manually added to their Gradescope course.

  • Either let students enroll themselves, by letting the students know about an entry code (Gradescope generates it: everyone can register with this code for your course)
  • or you as teacher manually upload the list of students as comma-separated values (CSV) format: just export in UGLA your student list in Microsoft Excel format, open in a spreadshet, export there as CSV and upload to Gradescope (double check that names containing special Icelandic characters are correct, i.e. try different CSV exports such as Unicode characterset).

Teacher adds students to course roster

Entry code for students to self-enroll (no video, screenshot only)

You can also add dæmakennarar/teaching assistants (TAs) to a course to let them grade using Gradescope: you just need to clarify who grades what or create separate courses for each TA. (While Gradescope supports the notion of sections=dæmatími groups, sections can currently only be set when populating the roster via CSV, but not web-based using entry code or a teacher later adding single students.)

Getting started

If you want give Gradescope a try, just go to https://www.gradescope.com, sign up (select University of Iceland and use your hi.is email address), create a dummy course and assignment. If you like, you can also add a dummy student using your private email address and play around.

The above features are only the most basic features of Gradescope, for more check:

For your info: IVT deild has paid for the Institution license, i.e. you have all features. (Except for the integration with Canvas that we can only do next semester when all course use Canvas.)

While we paid for 1500 students only, we are allowed to have as many students as we need in 2020 (in 2021, we might then have to pay for the number of students of 2020, so either HÍ as a whole adopts Gradescope or IVT deild convinces Gradescope that 2020 was exceptional -- they anyway started to give out free licenses because of COVID-19).

Computer Science department and DEEP-EST project at UTmessan 2020, Icelands biggest IT fair

Helmut Neukirchen, 10. February 2020

Our new colleague Morris Riedel gave on 7. February 2020 a presentation on Quantum Computing (slides / video) at UTmessan 2020, Icelands biggest IT fair. In addition, the Computer Science Department ran on the public visitor day (8. February 2020) a booth: beside student projects, we showcased research projects, e.g. DEEP-EST.

The DEEP-EST project

For showcasing the machine learning that we do in the DEEP-EST project, we offer a web page that allows you to use the camera of your smartphone (or laptop) to detect objects in real-time. While neural networks are still best trained on a supercomputer, such as DEEP-EST with its Data Analysis Module, the trained neural network even runs in the browser of a smartphone.

https://nvndr.csb.app/

Just open the following web page and allow your browser to use the camera: https://nvndr.csb.app/.

(Allow a few seconds for loading the trained model and initialisation.)

The used approach is Single Shot Detector (SSD) (the percentage shows how sure the neural network is about the classification) using the MobileNet neural network architecture. The dataset used for training is COCO (Common Objects in Context), i.e. only objects of the labeled object classes contained in COCO will get detected. The Javascript code that is running in your browser uses Tensorflow Lite and its Object Detection API and model zoo.

If you want learn more about DEEP-EST, have a look at the poster below (click on the picture below for PDF version):

PDF of DEEP-EST poster

Ranking Journals and conferences in Supercomputing and Data Science

Helmut Neukirchen, 22. November 2019

Many academics insist on that journals are better than conferences, e.g. some PhD programmes have unwritten rules that a PhD thesis needs to involve at least one journal publication (which can be really a problem, because some journals have 1.5 year time span from submission to publication; add this to another 1.5 year for a PhD student to produce the first results being worth published in a journal/top conference, then this is almost impossible in 3 years of PhD study).

For Computer Science, some conference are as hard (or even harder) as journals, e.g. in terms of acceptance rates (which however depends also a lot, e.g. having a lot of crap submissions automatically leads to a low acceptance rate). Also Computer Science is a very fast developing field, so results would be often outdated after 1.5 years, so the far shorter publication cycles make conferences far more attractive.

As an example, below are two rankings (based on impact, i.e. citations such as h-index) that show that Computer Science conference are as high-quality (or even higher) as journals. Of course, you can always find conferences (but also journals) that have a low impact: therefore, instead of claiming that in general journals are better than conferences, you always need to look at each particular conference, but also at each particular journal (acceptance rates are missing in these lists -- they would be nice to compare, but this data is tedious to collect):

Experiment on Google search results for Tölvunarfræði, Hugbúnaðarverkfræði, Reikniverkfræði, Computer Science, Software Engineering, Computational Engineering

Helmut Neukirchen, 11. November 2019

As the Department of Computer Science is hidden within the Faculty of Industrial Engineering, Mechanical Engineering and Computer Science, the visibility of the Department of Computer Science is somewhat hindered, in particular when navigating from the University of Iceland's home page.

The University of Iceland has some Icelandic and English web pages specific to our study programmes. The question to be investigated is whether at least a Google search for the Icelandic terms Tölvunarfræði, Hugbúnaðarverkfræði, Reikniverkfræði, and English terms Computer Science, Software Engineering, Computational Engineering yield the study programmes of the Department of Computer Science at University of Iceland.

Therefore, you find below the Google search results for these search words, using Google search (from an Icelandic client IP address which heavily influences the search results) on 11.11.2019. According to Google's page cache, https://uni.hi.is/helmut was last visited 8 Nov 2019 16:33:03 GMT, so the search results documented by screenshots below are most likely based on the that date, i.e. before this blog post was created.

If you read this page later, you are welcome to compare (assuming you browse with an Icelandic IP address as this influences Google's search results) whether the search results changed (to some extent this might then be due the links contained in this blog post and you might call this search engine optimisation).

Update 13.11.2019: According to Google's page cache, Google has visited on 12.11.2019 this blog entry and on 13.11.2019, the order of search results did not really change (except that for the search term Computer Science, the first hit that used to be a Wikipedia page disappeared and instead, a Feature Snippet appeared and thus, all other hits got one place better; for Software Engineering, the first two hits pointing to Wikipedia's entries for Software Engineer and Software Engineering swapped places). As the search results did not change significantly, I added therefore on 13.11.2019 a few more links pointing to University of Iceland study programme web pages.

Update 14.11.2019: According to Google's page cache, Google has visited this page again on 13.11.2019 (Google's crawling adapted to the frequency of updates of my page) and on 14.11.2019, however this seems to have been before the 13.11.2019 update. Still, the ordering for some search results has changed: For the search term Hugbúnaðarverkfræði, the University of Iceland's page on Hugbúnaðarverkfræði climbed up from 2nd to 1st place. Also for the search term Software Engineering, the University of Iceland PhD programme page got a push (even though it is not linked at all in this blog post).
One explanation might be that this blog seems to have a small influence Google's search results (the order for search term Hugbúnaðarverkfræði changed, but not for search term Computer Science). A single page having such an influence could be explained by the small number of web pages referring to the University of Icelandic web pages for the study programmes Tölvunarfræði/Computer Science, Hugbúnaðarverkfræði/Software Engineering, Reikniverkfræði/Computational Engineering.
Another explanation would be that the Google page ranking algorithm was changed at the same time. Future work would be to repeat this experiment with links to the respective PhD programme pages that are currently not linked at all in this blog post.

Search result for Tölvunarfræði

On 11.11.2019, the Google search for Tölvunarfræði gives as first hit the University of Iceland page for Tölvunarfræði that is linked in this blog post:


Search result for Hugbúnaðarverkfræði

On 11.11.2019, the Google search for Hugbúnaðarverkfræði gives as second hit the University of Iceland page for Hugbúnaðarverkfræði that is linked in this blog post:


Search result for Reikniverkfræði

On 11.11.2019, the Google search for Reikniverkfræði gives as second hit the University of Iceland page for Reikniverkfræði that is linked in this blog post:


Search result for Computer Science

(Note: scrolled beyond a Featured Snippet box.)

On 11.11.2019, the Google search for Computer Science gives as fourth hit (after scrolling) the University of Iceland page for Computer Science that is linked in this blog post:


Search result for Software Engineering

(Note: scrolled beyond a Featured Snippet, People also ask, and a Video box.)

On 11.11.2019, the Google search for Software Engineering gives as fourth hit (after scrolling) the University of Iceland page for Software Engineering that is linked in this blog post:


Search result for Computational Engineering

(Note: scrolled beyond a Featured Snippet and a Video box.)

On 11.11.2019, the Google search for Computational Engineering gives as second hit (after scrolling) the University of Iceland page for Computational Engineering that is linked in this blog post:

PhD Defense Standards-based Models and Architectures to Automate Scalable and Distributed Data Processing and Analysis

Helmut Neukirchen, 7. October 2019

Shahbaz Memon successfully defended his PhD thesis in Computer Science on Standards-based Models and Architectures to Automate Scalable and Distributed Data Processing and Analysis. The thesis covers Scientific Workflows and middlewares for High-Performance Computing and High-Throughput Computing.

PhD defense announcement

This PhD is an example of the collaboration between the Faculty of Industrial Engineering, Mechanical Engineering and Computer Science and Jülich Supercomputing Centre (JSC).

PhD candidate, opponents, dean, and PhD committee

Members of the PhD commitee were Morris Riedel, Helmut Neukirchen, and Matthias Book, opponents were Ramin Yahyapour and Robert Lovas. The head of faculty, Rúnar Unnþórsson, was steering the defense. While we have some on cultural diversity involved, we need to improve on gender diversity! More photos can be found on flickr.

European Researcher's Night: From the next generation supercomputer DEEP-EST to your smartphone -- real-time object detection using neural network

Helmut Neukirchen, 28. September 2019

The DEEP-EST research project is at Vísindavaka, part of the European Researcher's Night, in Reykjavik, 28. September 2019.

DEEP-EST Booth at European Researchers Night

Use the camera of your smartphone to detect objects in real-time. While neural networks are still best trained on a supercomputer, such as DEEP-EST with its Data Analysis Module, the trained neural network even runs in the browser of a smartphone. Bring your smartphone and objects such as apples, bananas or teddy bears to let your smartphone detect these objects.

https://nvndr.csb.app/

Just open the following web page and allow your browser to use the camera: https://nvndr.csb.app/.

(Allow a few seconds for loading the trained model and initialisation.)

The used approach is Single Shot Detector (SSD) (the percentage shows how sure the neural network is about the classification) using the Mobilenet neural network architecture. The dataset used for training is COCO (Common Objects in Context), i.e. only objects of the labeled object classes contained in COCO will get detected. The Javascript code that is running in your browser uses Tensorflow Lite and its Object Detection API and model zoo.

If you want learn more about DEEP-EST, have a look at the poster below (click for PDF version):

PDF of DEEP-EST poster

Research project European Open Science Cloud (EOSC)-Nordic starting

Helmut Neukirchen, 1. September 2019

University of Iceland was successful in a consortium applying for funding from the European Horizon 2020 research programme with the European Open Science Cloud (EOSC)-centric proposal EOSC-Nordic.

EOSC-Nordic aims to foster and advance the take-up of the European Open Science Cloud (EOSC) at the Nordic level by coordinating the EOSC-relevant initiatives taking place in Finland, Sweden, Norway, Denmark, Iceland, Estonia, Latvia, Lithuania, Netherlands and Germany. EOSC-Nordic aims to facilitate the coordination of EOSC relevant initiatives within the Nordic and Baltic countries and exploit synergies to achieve greater harmonisation at policy and service provisioning across these countries, in compliance with EOSC agreed standards and practices. By doing so, the project will seek to establish the Nordic and Baltic countries as frontrunners in the take-up of the EOSC concept, principles and approach. EOSC-Nordic brings together a strong consortium including e-Infrastructure providers, research performing organisations and expert networks, with national mandates with regards to the provision of research services and open science policy, and wide experience of engaging with the research community and mobilising national governments, funding agencies, international bodies and global initiatives and high-level experts on EOSC strategic matters.

A successful EOSC-Nordic will reinforce Nordic research area capability and competitiveness, create a profile of a leading knowledge based region, increase the ability of the region to attract talent and investments, enhance its appeal as a partner in cooperation, and strengthen the Nordic region and its efforts in the overall EOSC, through the creation of a cross-border cooperation model for Europe.

The project is coordinated by the Nordic e-Infrastructure Collaboration (NeIC) and the University of Iceland is one of the project participants. The University of Iceland's diverse team is lead by Ebba Þóra Hvannberg. Helmut Neukirchen and Morris Riedel contribute their knowledge with respect to e-Science, such as scalable, parallel machine learning, scientific workflows, and data federation. In addition to these researchers from the University's Computer Science department, experts from other departments of the University of Iceland contribute to EOSC-Nordic.

Project duration is 1st of September 2019 to 31st of August 2022. More information can be found on the EOSC-Nordic web page and also on my local page covering this research project.

EOSC Partners Group Photo

12th Nordic Workshop on Multi-Core Computing (MCC2019)

Helmut Neukirchen, 30. August 2019

The objective of MCC is to bring together Nordic researchers and practitioners from academia and industry to present and discuss recent work in the area of multi-core computing. This year's edition is hosted by Blekinge Institute of Technology in Karlskrona, Sweden.

The scope of the workshop is both hardware and software aspects of multi-core computing, including design and development as well as practical usage of systems. The topics of interest include, but is not limited to, the following:

Architecture of multi-core processors, GPUs, accelerators, heterogeneous systems, memory systems, interconnects and on-chip networks
Parallel programming models, languages, environments
Parallel algorithms and applications
Compiler optimizations and techniques for multi-core systems
Hardware/software design trade-offs in multi-core systems
Operating system, middleware, and run-time system support for multi-core systems
Correctness and performance analysis of parallel hardware and software
Tools and methods for development and evaluation of multi-core systems

There are two types of papers eligible for submission. The first type is original research work and the second type is work already published in 2018 or later.

Participants submitting original work are asked to send an electronic version of the paper that does not exceed four pages using the ACM proceedings format, http://www.acm.org/publications/proceedings-template, to https://easychair.org/conferences/?conf=mcc20190.

The same URL is to be used should you want to present an already published paper as described above. In that case, you need to clearly specify that the paper is already published and where the paper has been published.

No proceedings will be distributed. Contributions will not disqualify subsequent publication in conferences or journals.

The conference web page is https://sites.google.com/view/mcc2019.

Important dates

Sep. 29 2019: Submission deadline
Oct. 27 2019: Author notification
Nov. 18 2019: Registration deadline
Nov. 27-28 2019: MCC Workshop

PhD Defense GraphTyper: A pangenome method for identifying sequence variants at a population-scale

Helmut Neukirchen, 26. June 2019

Hannes Pétur Eggertsson successfully defended his PhD thesis in Computer Science on GraphTyper: A pangenome method for identifying sequence variants at a population-scale. I had the honor to steer this defense in my role as vice head of faculty.

As you notice, only men are occurring here. We need to improve on this! More pictures can be found on flickr.

Datasets for DBSCAN evaluation

Helmut Neukirchen, 20. June 2019

For evaluating implementations of the popular DBSCAN clustering algorithm, various publications use several datasets. Pointers to these datasets and information on paramaters (e.g. normalisation, epsilon and minpts) are collected here. You are welcome to contact me if you have further (big) datasets that are good benchmarks for DBSCAN.

Sarma et al.: μDBSCAN: An Exact Scalable DBSCAN Algorithm for Big Data Exploiting Spatial Locality

TODO: check in detail datasets used, but some are those datasets used in some of the other publications below, but "In addition, we have also used a few other real datasets: 3D Road Network (3DSRN) [32] contains vechicular GPS data; Household Power (HHP*) and KDDBIO145K (KDDB*) datasets are borrowed from UCI Repository [33]."

Gan, Tao: DBSCAN Revisited: Mis-Claim, Un-Fixability, and Approximation

Data normalized to [0, 10^5 ] for every dimension.

MinPts = 100, Epsilon = 5000 and higher. (Note: far too high value turning almost the entire dataset into a single cluster -- the mis-claim is on their side!).

Their preprocessed datasets

  • PAMAP2 (3,850,505 4D points),
  • Farm (3,627,086 5D points),
  • Houshold (2,049,280 7D points)

can be obtained from their webpage.

Mai, Assent, Jacobsen, Storgaard Dieu: Anytime parallel density-based clustering

  • Same household datasets used as by Gan, Tao.
  • Also PAMAP2 is used, but claimed to be 974,479 39D points whereas Gan and Tao reduced it to 4 dimensions using PCA, but claim to have 3,850,505 points.
  • In addition, the UCI Gas Sensor dataset by Fonollosa et al. is used: 4,208,261 16D points (DETAILS NOT PROVIDED IN PAPER).

Kriegel, Schubert, Zimek: The (black) art of runtime evaluation: Are we comparing algorithms or implementations?

  • Same PAMAP2, Farm and household datasets used as by Gan, Tao (including also smaller epsilon values as these make more sense).
  • In addition, for higher dimensional data, the Amsterdam Library of Object Images (ALOI) dataset from Geusebroek et al is used, namely the 110250 HSV/HSB color histograms provided on the ELKI Multi-View Data Sets webpage. Namly, the eight dimensions (two divisions per HSV color component) dataset (I assume, this is the 2x2x2 dataset) with epsilon=0.01 and minPts=20.

Patwary, Satish, Sundaram, Manne, Habib, Dubey: Pardicle: parallel approximate density-based clustering

PDSDBSCAN

A subsampled version of the above Millenium Run dataset has also been used in the paper A new scalable parallel DBSCAN algorithm using the disjoint-set data structure by the same main author as Pardicle describing and evaluating PDSDBSCAN who published also a 50,000 10D point dataset used also in that paper.

Götz, Bodenstein, Riedel: HPDBSCAN: highly parallel DBSCAN

The Bremen 3D point cloud and Twitter 2D GPS locations are available as full and subsampled (small) datasets: DOI: 10.23728/b2share.7f0c22ba9a5a44ca83cdf4fb304ce44e (Note: the original publication refers to the dataset via a handle.net handle which does not work anymore).

  • Twitter (dataset t): 16,602,137 2D points (eps=0.01, minPts=40). Note that this dataset contains some bogus artefacts (most likely Twitter spam with bogus GPS coordinates).
  • Twitter small (dataset ts): 3,704,351 2D points (eps=0.01, minPts=40)
  • Bremen (dataset b): 81,398,810 3D points (eps=100, minPts=10000)
  • Bremen small (dataset bs): 2,543,712 2D points (eps=100, minPts=312)

Neukirchen: Elephant against Goliath: Performance of Big Data versus High-Performance Computing DBSCAN Clustering Implementations

The same Twitter small dataset as provided by Götz et al. has been used with the same parameters.