AWS and Unix intro

Connecting and properly using a cloud computing cluster at the CBW

by Jose Hector Galvez, Zhibin Lu & Rob Syme

Schedule:

Today’s schedule can be found here.

Contents:

  1. Logging into AWS

  2. Introduction to the command line

    2.1 Exercise: Exploring the filesystem

  3. File manipulation

    3.1 Exercise: Reading text files

    3.3 Exercise: Editing text files

  4. Searching and sorting files

  5. Putting it all together

6.AWS Machine Image (AMI)

1. Logging into AWS

Description of the lab:

This section will show students how to login to AWS and create an instance.

Once you have confirmed your account as per the email (subject line “You have been invited to join an AWS Educate Classroom”), you can log in here:

2. Introduction to the command line

Description of the lab:

This section will show students the basics of the command line, with a focus on navigation.

Exercise: Exploring the filesystem

  1. Connect to your AWS instance
  2. Type the ls command, what do you see?
Solution (click here) ``` $ ls CourseData R cvmfs_cache workspace ``` The `ls` command lists the contents of a working directory.
  1. After following the tutorial, can you answer what these commands (cd, pwd) do?
Solution (click here) The `cd` command is used to *change directories*. Without arguments, it will move to the home directory (`~`) The `pwd` command shows the absolute *path to the working directory*.

3. File manipulation

Description of the lab:

This section will show students how to manipulate files, including reading, editing, and renaming text files.

Additional material:

Here are two cheat-sheets that can be useful to have as a reference for common UNIX/Linux commands:

Exercise: Reading text files

  1. What do the commands cat, head, and tail do? What do they have in common?
Solution (click here) All three of these commands ouptut the contents of a text file to *standard out*: - `cat` outputs the *full* contents of the file - `head` outputs the *first* 10 lines of a file - `tail` outputs the *last* 10 lines of a file
  1. What does the command less do? How is it different from cat?
Solution (click here) `less` opens a text file for viewing. Unlike `cat`, it will display it in a separate file viewer.
  1. How can you know the number of lines in a file?
Solution (click here) The command `wc -l` will display the number of lines in a file. `wc` (word count) displays the number of words, lines, and bytes in a file. The `-l` option, limits the output to lines.

Exercise: Editing text files

  1. Write “Hello world” into a file called helloworld.txt using nano. Save and then exit.
Solution (click here) First, use the `nano` command to open a file called `helloworld.txt` ``` $ nano helloworld.txt ``` Inside the nano editor, write "Hello world" and then use the `^O` option to write the changes and then `^X` to exit.
  1. Create a subdirectory called test. Then, move the helloworld.txt file into the directory.
Solution (click here) First, use the command `mkdir` to create this new directory. Then, use `mv` to move `helloworld.txt` into this directory. ``` $ mkdir test $ mv helloworld.txt test/ ```
  1. Create a copy of the helloworld.txt file called helloworld2.txt, inside the test directory.
Solution (click here) First, change the working directory using `cd`, then use the `cp` command to create the copy. ``` $ cd test $ cp helloworld.txt helloworld2.txt ```

4. Searching and sorting files

Description of the lab:

This section will show students how to search for files and in files.

Workshop notes and quiz questions here.

5. Putting it all together

Description of the lab:

This section will show students how the basic concepts fit together and in the context of bioinformatics.

Workshop notes and quiz questions here.

6. AWS Machine Image (AMI)

Starting the AWS AMI after the workshop is over:

An updated AWS Machine Image (AMI) will be be prepred by Zhibin at the end of this CBW workshop: It will have the same software used in the workshop, but it will not contain any of the data. To get such an image started, you will need to set up your own AWS account with your own credit card, and then, the same as you did in this workshop, look for the CBW AMI, and start that with your own project.

Launching the AMI

Let us know on the slack channel or at course_info@bioinformatics.ca if you are experiencing any problems with this.