CSC8636 – Summative Assessment
Visual Analysis of the Ocean Microbiome
Background
Data visualization has become an important tool for explorative data analysis as well as for presentation and communication of data in many application domains. A domain that has become increasingly data driven over the last decades are biosciences, and in particular when it comes to studies of the microbiome and other genome sequenced data. In this summative assessment, you are asked to design and implement an interactive multiple coordinated views visualization that support analysis of data from a study of the ocean microbiome, using different visualization methods.
The focus of the tasks in the assessment is on visualization of heterogeneous and multivariate (high dimensional) data, interactive visualization and multiple views, heuristic evaluation, and visualization of uncertainty.
Data context
The oceans are the largest cohesive eco-system on earth, and a greater understanding of this eco-system is important for the preservation of the planet as well as for understanding of how organisms have evolved since life began. The data that you will work with originates from a two-and-a-half-year expedition with the schooner Tara, during which oceanic samples were collected from 210 stations across the world oceans. If you are interested, you can read more about the expedition and ocean microbiome here: https://www.embl.org/topics/tara/
User context
The end user of the visualization that you will develop would typically be a microbiologist or another domain expert in a bioscience field. The aim of their analysis would be to increase their knowledge of the ocean microbiome, and analysis questions of particular interest may for example include:
• Which microbes are detected at the highest levels overall in the oceans?
• Which microbes are detected at the highest levels in certain regions of the oceans?
• Are there differences in microbe detection levels that can be linked to other features of the oceanic samples, for example the geographic region, sample depth etc?
• Are there differences between taxonomic levels, which can be linked to other features of the oceanic samples?
The data
You will be provided with a set of different spreadsheets to work with, which have gone through some initial formatting and cleaning. The full dataset include data related to 135 samples that were taken from different oceanic regions.
The detection levels of 35,650 Operational Taxonomic Units (OTUs) were recorded for the individual samples. Detection levels are sometimes referred to as the abundance of the OTU. OTUs are close approximations of microbial species, which are extracted through clustering of DNA sequences, so you can think of an OTU as being the same as a microbial species (such as a bacterium) . The OTUs also have an associated hierarchical taxonomy through the biological classification system (https://en.wikipedia.org/wiki/Taxonomy_(biology)), and are often converted into higher levels in the taxonomy for analysis, since an OTU name generally has no biological meaning. Analysis is quite often carried out and reported at Genus level.
In addition to the OTU detection levels, there area range of contextual data associated with the samples (i.e. metadata) . From a data science and visualization perspective, the OTUs are generally treated as data variables (dimensions) and the samples are data items.
You will be provided with the following datasets, in comma separated file format (csv):
• Tara_OTUtableTax_full.csv: Each row in this file corresponds to a unique OTU (microbial species). The first six columns include the taxonomic classification for each OTU at the following hierarchical levels: Domain, Phylum, Class, Order, Family, Genus. The original taxonomic classification of the OTUs included a lot of missing values, as a result of OTUs that were not identifiable at all levels in the taxonomy. The highest level where nearly all OTUs were identified was the Class level. Due to this, the missing values have been replaced with the Class name of the OTU, followed by (undef) (i.e. a Cyanobacteria OTU is referred to as Cyanobacteria(undef) at all levels where it has not been classified). The seventh column include a unique OTU-id, which has no biological meaning. The remaining columns each correspond to a sample, with a unique sample id as heading. The cells represent the relative detection level (relative abundance) of OTUs in samples as a percentage value, thus the sum of each column is 100%.
• Tara_OTUtableTax_80CAb.csv: This file includes a subset of the same data as Tara_OTUtableTax_full.csv. It is reduced to include only the 1400 most abundant OTUs, which make up 80% of the total cumulative abundance of the full dataset.
• Tara_OTUtableTax_80Cab_transp.csv: This file includes a transposed version of Tara_OTUtableTax_80Cab.csv, without the taxonomy. In this dataset each row represent a sample and each column represent an OTU, with the first column representing the sample id.
• Tara_SampleMeta.csv: Each row in the file correspond to a sample, with sample id’s that are identical to those in the OTU tables. The columns include contextual information about the samples, including: SampleID, Year, Month, Latitude[degreesNorth], Longitude[degreesEast], SamplingDepth[m], LayerOfOrigin, MarinePelagicBiome, OceanAndSeaRegion, MarinePelagicProvince.
You can choose yourself which version of the OTU table to use, and are welcome to perform any data wrangling or modification using a tool of your choice prior to visualization.
The assignment
The coursework consists of three parts, which are detailed below. Submission and implementation details are provided at the end of the document.
Part 1: Interactive visualization using multiple coordinated views (60%)
The first and main task of the coursework is to design and implement an interactive multiple coordinated views visualization that support exploration of the Tara Ocean data, using one of the OTU tables and the sample metadata. The final multiple coordinated views visualization should be saved and submitted as an html page.
The aims of the visualization are to:
1. Help the user understand overall abundance patterns and diversity in the oceans: the user would typically be interested in knowing which the most abundant microbes are, and if there are large variations in abundance between different microbes.
2. Help the user understand some of the abundance patterns and diversity in the oceans in context of the sample meta data: e.g. Are there differences in abundance profiles between different sample classes? What does such differences tellus about the Ocean microbiome in context of the sample categories?
3. Help the user get an overview of the microbiome while also being able to investigate details and patterns of potential interest in more detail: a user may, for example, be interested to know if there are differences between different taxonomic levels, to identify and explore patterns that are visible only in subsets of data, or to compare specific subsets of samples in more detail.
For full marks you are expected to include at least three views in your visualization, which are interactively coordinated and display different aspects of the data. You are also expected to take accessibility and user diversity into consideration.
Fill in the relevant parts of the submission table to demonstrate your approach to meeting the aims. You need to demonstrate in the table your use of visualization theory and principles in the design, and to justify design choices made. You are expected to also reflect on alternative visualization approaches and methods, and how these could have been used.
Part 2: Uncertainty in data (10%)
Based on your visualization in part 1: Reflect on potential sources of uncertainty in the data, and how you could approach visualizing them. You do not have to implement anything for this but fill in the relevant part of the submission table.
Part 3: Heuristics evaluation (20%)
Based on your visualization in part 1: Reflect on how the visualization meet the visualization heuristics of Wall et al. (2019), and how you could modify the visualization to better meet these heuristics. You do not have to implement anything for this and should not carryout an evaluation with other participants but fill in the relevant part of the submission table.
Note: marking in this section is not based on if you meet the heuristic criteria, but on your understanding of how the heuristics could be met. Hence, not meeting a heuristic criterion but having a good suggestion of how you could meet it may be marked equally high as meeting the heuristic.
Use of language/tools
You must use Python and are recommended to use the Altair and Pandas packages for creating the interactive multiple coordinated views visualization in part 1. You are allowed to use other Python visualization packages, although there will be limited technical support for them and you must make sure you are able to generate an html version of the multiple coordinated views visualization.
You are free to use any language or software of choice for any data wrangling that you need to do. Make sure to detail in the appendix and reference list in your submission which packages and software you have used (Python and non-Python).
What to submit
Coursework
• Submit in Canvas a single zip file including:
o Report: A document including the submission table with details and justification of your visualization and design choices, and a list of references to sources used to carry out the project, in pdf format. References in the report must be consistently cited in a standard way.
o Visualization: The html page with your multiple coordinated views visualization from part 1 (note : this should not bean html version of a Jupyter Notebook, but an html-file saved using Altair’s ‘save’ functionality or similar).
o Code: Your Python code and the datasets that are loaded by the code.
The coursework submission deadline is 16:30 on Thursday 22rd February.
Oral presentation
Submit in Canvas a short (5-7 min) video demonstration of your visualization and its interactive features. The videos will be shared with others in the module when all have submitted. Video recordings can be made using, for example, Zoom or Microsoft Teams, by recording a meeting where you share your screen.
請加QQ：99515681 郵箱：99515681@qq.com WX：codehelp
