TAKE 2 CLASSES A DAY FOR A WEEK

Transcription

1 Only administer as prescribed abuse outside of seminar may lead to UNIX and Pyhton addiction for which no remedy is known Meeting schedule Monday May 6 to Friday May 10, 10am-noon, 2-4 in Dirac 152. This course has no prerequisite but may need new thinking and follow instructions, the form of the course is unclear and not well defined yet. We will learn about basic UNIX commands and basic Python commands to allow manipulation of files and perhaps are able to construct pipelines to transform data from one format into another SC/ remedial Scientific Computing DSL 152 You and your friends Florida State University UNIX and Python 120 MIN Common brands: macosx, linux TAKE 2 CLASSES A DAY FOR A WEEK Qty: 10 Refills: as needed Phone: (850) RX # Prescriber: PETER BEERLI

2 Workshop about Python and basic UNIX within a short week Week May 6-10 Monday UNIX 2-4 UNIX Tuesday Introduction to Python; what is it and why should you use it 2-4 Hello world and strings/words/texts Wednesday using modules and writing more complex program 2-4 File input/output Thursday Stringing things together 2-4 Plotting data Friday Exploration of modules 2-4 Outlook, Review. We have a total of 18 seats! Register sending to beerli@fsu.edu

3 Today Overview of UNIX First contact Details of most important commands Editing in the UNIX commandline Practice Pipelines Extracting the stuff you want Synthesis

4

5

6

7

8

9

10 A UNIX command line consists of the name of a UNIX command (actually the "command" is the name of a built-in shell command, a system utility or an application program) followed by its "arguments" (options and the target filenames and/or expressions). The general syntax for a UNIX command is $ command -options targets Here command can be though of as a verb, options as an adverb and targets as the direct objects of the verb. In the case that the user wishes to specify several options, these need not always be listed separately (the options can sometimes be listed altogether after a single dash). William Knottenbelt (2001)

11 1. Log on a Linux machine or connect to one from a Windows machine (e.g. click on the Exceed icon and then use putty to connect to the server kiwi. Enter your login (user name) and password at relevant prompts. 2. Enter these commands at the UNIX prompt, and try to interpret the output. Ask questions and don't be afraid to experiment (as a normal user you cannot do much harm): echo hello world passwd date hostname arch uname -a dmesg more (you may need to press q to quit) uptime who am i who id last finger w top (you may need to press q to quit) echo $SHELL echo {con,pre}{sent,fer}{s,ed} man "automatic door" man ls (you may need to press q to quit) man who (you may need to press q to quit) who can tell me why i got divorced lost clear cal 2000 cal (notice anything unusual?) bc -l (type quit or press Ctrl-d to quit) echo 5+4 bc -l yes please (you may need to press Ctrl-c to quit) time sleep 5 history William Knottenbelt (2001)

12 UNIX editors Most common basic UNIX editors

13 Pipelines stdin stdout stderr

14 cat hello.txt sort uniq cat hello.txt grep "dog" grep -v "cat" To redirect standard output to a file instead of the screen, we use the > operator: $ echo hello hello $ echo hello > output $ cat output hello In this case, the contents of the file output will be destroyed if the file already exists. If instead we want to append the output of the echo command to the file, we can use the >> operator: $ echo bye >> output $ cat output hello bye

15 To capture standard error, prefix the > operator with a 2 (in UNIX the file numbers 0, 1 and 2 are assigned to standard input, standard output and standard error respectively), e.g.: $ cat nonexistent 2>errors $ cat errors cat: nonexistent: No such file or directory $ You can redirect standard error and standard output to two different files: $ find. -print 1>errors 2>files or to the same file: or $ find. -print 1>output 2>output $ find. -print >& output Standard input can also be redirected using the < operator, so that input is read from a file instead of the keyboard: $ cat < output hello bye You can combine input redirection with output redirection, but be careful not to use the same filename in both places. For example: $ cat < output > output will destroy the contents of the file output. This is because the first thing the shell does when it sees the > operator is to create an empty file ready for the output.

16 Grep

17 Common File System Commands ls! List names of all files in current directory ls filenames! List only the named files ls -t! List in time order, most recent first ls -l! Long listing, more information. Also ls -lt! ls -t! List by time last used. Also ls -lu, ls -lut! ls -r! List in reverse order. Also ls -rt, ls -rlt, etc! ed filename! Edit named file cp file1 file2! Copy file1 file2. Overwrite old file2 if it exists mv file1 file2! Move file1 file2. Overwrite old file2 if it exists rm filenames! Remove named files, irrevocably cat filenames! Print contents of named files pr filenames! Print content with header, 66 lines per pages (default) pr -n filenames! Print in n columns pr -m filenames! Print named files side by side in multiple columns wc filenames! Count lines, words, and characters for each file wc -l filenames! Count lines for each file grep pattern filenames! Print lines matching pattern grep -v pattern filenames! Print lines not matching pattern sort filenames! Sort files alphabetically by line tail filename! Print last 10 lines of file tail -n filename! Print last n lines of file tail +n filename! Start printing file at line n cmp file1 file2! Print location of first difference diff file1 file2! Print all differences between files

18 Shell Metacharacters >! prog > file direct standard output to file >>! prog >> file append standard output to file <! prog < file take standard input from file! p 1 p 2 connect standard output of p 1 to standard input of p 2 <<here! here document: standard input follows, up to next here on a line by itself! *! Match any string of zero or more characters in filenames!?! Match any single character in filenames [ccc]! Match any single character from [ccc] in filenames. Ranges like 0-9 or a-z are legal ;! Command terminator: p 1 ; p 2 does p 1, then p 2 &! Like ; but does not wait for p 1 to finish ` `! ( )! { }! Run command(s) in ; output replaces ` `! Run command(s) in in a sub-shell! Run command(s) in in current shell (rarely used) $1, $2, etc $0 $9 replaced by arguments to shell file $var! ${var}! Value of shell variable var Value of var; avoids confusion when concatenated with text \! \c take character c literally, \newline discarded!! Take literally Take literally after $, ` ` and \ interpreted #! Text after # is a comment var=value! Assign value to variable var p 1 && p 2 Run p 1 ; if successful, run p 2 p 1 p 2 Run p 1 ; if unsuccessful, run p 2

19 Shell I/O Redirections > file direct standard output to file >> file append standard output to file < file take standard input from file p 1 p 2 connect standard output of program p 1 to input of p 2 ^! obsolete synonym for! n>file n>>file direct output from file descriptor n to file! append output from file descriptor n to file! n>&m merge output from file descriptor n with file descriptor m! n<&m merge input from file descriptor n with file descriptor m! <<s <<\s << s here document: take standard input until next s at beginning of a line; substitute for $, ` `, and \ here document with no substitution here document with no substitution

20 grep and egrep Regular Expressions (decreasing order of precedence) c any non-special character c matches itself \c turn off any special meaning of character c ^! beginning of line! $! end of line!.! any single character! [ ]! any one of characters in ; ranges like a-z are legal! [^ ]! any single character not in ; ranges are legal! \n what the n th \( \) matched (grep only) r* zero or more occurrences of r r+ one or more occurrences of r (egrep only) r? zero or more occurrences of r (egrep only) r 1 r 2 r 1 followed by r 2 r 1 r 2 r 1 or r 2 (egrep only) \(r\)! tagged regular expression r (grep only); can be nested (r)! regular expression r (egrep only); can be nested No regular expression matches a newline.

21 Shell Built-in Variables $# the number of arguments $* all arguments to shell. $* is a single word! $@ similar to $*. $@ is identical to the list of the arguments to shell $- options supplied to the shell $? return value of the last command executed $$ process-id of the shell $! process-id of the last command started with & $HOME default argument for cd command $IFS list of characters that separate words in arguments $MAIL file that, when changed, triggers you have mail message $PATH list of directories to search for commands $PS1 prompt string, default $ $PS2 prompt string for continued command line, default >

22 Shell Pattern Matching Rules *! match any string, including the null string!?! match any single character [ccc]!! \c! a b! match any of the characters in ccc [a-d0-3] is equivalent to [abcd0123] match exactly; quotes protect special characters. Also match c literally in case espressions only, matches either a or b / in filenames, matched only by an explicit / in the expression; in case, matched like any other character. as the first character of a filename, is matched only by an explicit. in the expression