B2.5 File processing

Text files are the simplest and most widely shared kind of file. Spreadsheets export their tables as comma-separated text, web servers record every request as one more line in a log, and settings files, source code and data sets for machine learning are all stored as plain text. A program that can read such a file line by line, split each line into its parts and write results to a new file can exchange data with almost any other program.

Files and persistent storage

Why programs need files

While a program runs, its variables, lists and arrays live in random-access memory (RAM). RAM is volatile: its contents are lost when the program ends or the power goes off. Data that must survive has to be copied to secondary storage, such as a solid-state drive or a hard disk drive, which is non-volatile.

Secondary storage is organised into files, which the operating system (OS) keeps track of through its file system. A program never writes directly to the surface of a disk. It asks the OS to open a named file, and the OS finds where the file's data is stored, checks that the program is allowed to use it and passes data between the device and the program.

Text files, lines and newline characters

A file is a sequence of bytes. In a text file, every byte or group of bytes stands for a character according to a character set, usually UTF-8, so the file can be opened and read in any text editor. Other files, such as photos, music and compiled programs, are binary files: their bytes are numbers or machine instructions with no meaning as characters, and an editor shows them as nonsense.

A text editor shows a text file as lines one below another, but on disk there are no lines, only one long row of characters. The end of each line is marked by an invisible character that tells the editor, and the program, to start a new line.

The diagram of scores.txt shows the same file twice: as an editor displays it, and as the 41 characters actually stored. Each of the five lines ends in a newline character, so "Amara,72" is 8 visible characters plus 1 newline, 9 in total. The positions under the cells are counted from 0, like string indices.

The program read_whole.py confirms the count. It opens the file, reads every character into one string and prints the string and its length.

Run against scores.txt, both versions print the five lines followed by a blank line and the count. The blank line appears because the last character of the file is a newline, and print (or println) then adds one of its own.

Records, fields and delimiters

Most data files are organised the same way as a database table. Each line holds one record, the data about one item, and each record is split into fields. In scores.txt, each line is one student's record and each record has two fields, a name and a score. A character that never appears inside the data, here the comma, separates the fields.

The delimiter must be a character that cannot occur in a field. A comma works for names and scores, but not for addresses ("12 High Street, Leeds") or numbers written with thousands separators. Real CSV files solve this by putting such fields in double quotes, as the common description of the format sets out.

Source: RFC 4180, "Common Format and MIME Type for Comma-Separated Values (CSV) Files", October 2005.

The last rule is why a simple split on commas is not enough for every real CSV file: "Smith, Jo",88 would be split into three pieces. For files whose fields never contain the delimiter, such as scores.txt, splitting on the delimiter is correct; Python's csv module and similar libraries exist for files that follow the full rules.

Sequential access

A text file is read in the same direction as a person reads a page: from the first character to the last. To reach the fourth line, a program must read past the first three, because lines have different lengths and there is no way to calculate where line 4 starts without counting the characters before it.

Sequential access fits tasks that use every record anyway: totalling, counting, finding the highest value, copying selected records to a new file. Finding one record is a linear search, so the time it takes grows in proportion to the number of records. Changing one record in the middle is harder still: the usual method is to read the whole file, change the data in memory and write the whole file again. Systems that look up and update single records many times a second use a database instead.

File paths

A program names a file by its path. A name such as "scores.txt" on its own is a relative path: the OS looks for it in the program's current working directory, usually the folder the program was started from. A path that starts from the top of the file system, such as /home/amara/school/scores.txt on Linux and macOS or C:\Users\Amara\school\scores.txt on Windows, is an absolute path and points to one place whatever the working directory.

Relative paths are a frequent source of errors. A program that works when started from its own folder fails with "file not found" when an IDE runs it from a different working directory, although the file has not moved. In a Java string, a Windows backslash must be doubled ("C:\\data\\scores.txt") because a single backslash starts an escape sequence such as \n; forward slashes work on every system in both languages.

How a program works with a file

Opening a file and the file handle

Before a program can use a file, it must open it. Opening asks the OS to find the file, check the program's access rights, prepare memory for transferring data and, for writing, create the file if necessary. The OS then gives the program a reference through which every later operation goes.

A file handle holds none of the file's data. It works like a library ticket, which identifies a borrowed book without containing it: the handle identifies one open file, and the OS records against it the program's current position in that file. The OS keeps a table of every file each program has open, and each entry uses memory, so a program can hold only a limited number of files open at once.

The file pointer

Each open file has a current position, the place where the next read or write will happen. When a file is opened for reading, the position starts at character 0. Every read moves it forward past the characters just read, which is why two calls to the same read operation return different lines.

The arrows in the diagram of scores.txt mark the file pointer at positions 0, 9 and 16 as the first two lines are read, and at position 41 when every character has been read. Python can report the position with file.tell(); reading scores.txt line by line gives the positions 9, 16, 26, 33 and 41.

Buffers and flushing

Reading or writing secondary storage is many thousands of times slower than working in RAM, and a device transfers data most efficiently in large blocks. So the OS and the language's file classes do not fetch one character at a time. When a program reads the first line, a whole block of the file, several thousand characters, is copied into an area of RAM, and later reads take their lines from there.

The diagram of buffered reading and writing shows both directions. On the reading side, the OS copies the file into the buffer (1) and each readline() takes the next line from it (2). On the writing side, write() only places text in the buffer (3); nothing reaches the disk until the buffer is full, the program asks for it to be emptied, or the file is closed (4).

Buffering makes file processing fast, but it means written data is not safe until it has been flushed. The program buffer_demo.py writes a line, reads the same file through a second handle before and after closing the first, and prints what the second handle finds.

Before the writer is closed, the file on disk is still empty, so Python reads an empty string (repr shows it as '') and Java's readLine() returns null. Closing the writer flushes its buffer, and the second reader then finds the saved line.

The Java version prints Before close: null and After close: Saved line. The same effect causes real data loss. A Java program that writes a line with FileWriter and ends without calling close() leaves a file of 0 bytes, because the text was still in the buffer when the program stopped. Python usually flushes open files when a program ends normally, but loses buffered data if the program crashes or is stopped.

Opening a file: read, write and append modes

A program states what it intends to do with a file when it opens it. The choice decides whether the program may read or write, what happens to data already in the file, where the file pointer starts and what happens if the file does not exist.

The diagram of the three modes shows the same two-line file opened each way, then the effect of writing the record Dev,91. Write mode keeps only the new line; append mode keeps the old lines and adds the new one after them; in read mode the attempt to write fails with an error.

Write mode empties the file at the moment it is opened, before the first write() runs. A program that opens an existing file in "w" mode and then crashes, or never writes, still leaves the file empty. That is the reason to open in write mode only when the whole file is meant to be replaced.

Opening in Python

Python uses one built-in function, open(path, mode), for every mode. It returns a file handle with methods for reading (read(), readline(), readlines()), writing (write()) and closing (close()). If the mode is left out, it is "r".

Two further details complete the picture. Python also has an "x" mode, which creates a new file and fails if one already exists, and modes with + that allow reading and writing together; neither is needed for sequential processing. When no character encoding is given, Python 3.15 (October 2026) and later use UTF-8 on every operating system. Versions up to 3.14 use the computer's locale encoding instead, which on many Windows computers is not UTF-8, so programs that must run on older versions and share files between computers add encoding="utf-8" to the call.

Opening in Java

Java has no single open function. A program opens a file by creating an object of a class designed for one job, and the choice of class is the choice of mode.

The second argument of FileWriter is a boolean: true means append, and leaving it out (or passing false) means overwrite. The classes live in the packages java.io (File, FileReader, BufferedReader, FileWriter, IOException) and java.util (Scanner), so a program must import them. Since JDK 18 (March 2022), these classes use UTF-8 by default on every operating system.

The program modes_demo.py shows the difference directly. It opens a file in write mode twice, writing one line each time, then does the same with a second file in append mode, and prints the lines in each.

The second write-mode opening wiped out "run 1", while append mode kept both lines. The Java version prints w_mode.txt [run 2] and a_mode.txt [run 1] [run 2].

Reading from a text file

Python and Java offer three ways to read a text file: all of it at once, one line on request, or every line in turn with a loop. All three move the file pointer forward, and all three must be preceded by opening the file and followed by closing it.

Reading the whole file at once

Python's read() returns every character from the file pointer to the end as a single string, newlines included, as read_whole.py showed. It is the simplest method when the program needs the text as one block: to count characters, to search for a word anywhere, or to display the file. Its drawback is memory: a 2 GB log file read with read() needs at least 2 GB of RAM for the string. Java's Scanner and BufferedReader have no single method in the course for reading a whole file, so a Java program builds the string from its lines, as the Java version of read_whole.py does. Each nextLine() drops the newline, so the loop adds "\n" back to rebuild the original text.

A second call to read() does not start again from the beginning. The first call moved the file pointer to the end, so there is nothing left to read, and the program read_twice.py shows the result.

The first read returns all 41 characters and the second returns an empty string. In Java, the first loop counts 5 lines and the second loop counts 0 (output 5 0).

Reading one line at a time

Python's readline() returns the characters from the file pointer up to and including the next newline, then leaves the pointer at the start of the following line. Java's BufferedReader.readLine() and Scanner.nextLine() do the same, with one difference that matters: Java removes the newline from the string it returns, Python keeps it. The program read_lines.py reads the first two lines of scores.txt and then examines the first one.

In the Python output, each name is followed by a blank line: the newline kept at the end of the string, plus the one print adds. The first line is 9 characters long with its newline and 8 after strip(), which removes spaces, tabs and newlines from both ends of a string.

The Java version prints no blank lines, gives the length as 8, and prints true, because readLine() has already dropped the newline.

Reading every line until the end of file

Most file processing reads every record. The number of lines in the file is not known in advance, so the program cannot use a counted loop; it uses a conditional loop that keeps reading until there is nothing left.

No special character is stored at the end of a text file. The end of file is a state, reached when there are no more characters to read. Each language reports it in its own way, and the loop condition must test for exactly that signal.

The table explains why Python can test for an empty string and Java cannot. In Python, even an empty line in the middle of a file comes back as "\n", so "" can only mean the end of the file. In Java, an empty line comes back as "", so the end of the file needs the different value null, which is the absence of any string.

The flowchart of the reading loop shows the pattern used by both languages: read one line before the loop (a priming read), test whether it signals the end of the file, and at the bottom of the loop read the next line before testing again.

The program read_until_eof.py turns the flowchart into code: it numbers and prints every line, then reports how many there were.

Both versions print the same output for scores.txt.

The shorter loops: for line in file and hasNextLine()

Python lets a for loop traverse a file handle directly: for line in file: reads one line per pass, with its newline, and stops at the end of the file without any test written by the programmer. Java's Scanner gives the matching pattern with hasNextLine(), which checks whether another line exists before nextLine() reads it.

Both print the five lines of scores.txt, one per line. The while line != "" and while (line != null) loops remain useful, because they show the end-of-file test explicitly, and that test is what a trace table of the loop records.

Loading a file into a list

Some tasks need the records more than once, for example to compute an average and then list everyone below it. Reading the file once into a list, and then working on the list, avoids reading the file twice. Python's readlines() returns every remaining line as a list of strings; in Java, an ArrayList<String> is filled one line at a time.

The Python output shows the newlines that readlines() keeps and the clean strings after strip(); the Java list prints as [Amara,72, Bao,85, Chiara,64, Dev,91, Elif,78].

Loading everything into memory is fast and convenient for a class list, but uses memory in proportion to the size of the file. For very large files, such as a server log with millions of lines, processing one line at a time keeps memory use constant, because only the current line is held.

Choosing Scanner or BufferedReader

Java's two reading classes overlap. Scanner is designed to break input into pieces: besides nextLine(), it has nextInt(), nextDouble() and next() (one word), and the same class reads from the keyboard (new Scanner(System.in)). BufferedReader does one job, reading lines quickly through a large buffer.

For small files of records, either class works. A Scanner is the shorter choice when lines contain numbers to read; a BufferedReader is the usual choice for long files, and its readLine() loop matches Python's readline() loop step for step.

Processing records: parsing and type conversion

Splitting a line into fields

A line read from a file is a single string, whatever it contains. To use a record, the program must divide the line into its fields and convert each field to the right data type.

Both languages split a string with a method called split, which takes the delimiter and returns the fields in order. Python returns a list and Java returns an array of String, indexed from 0 in both.

The final step is type conversion. The field "85" is a string of two characters, not a number, so adding it to a total either fails (Python raises a TypeError when adding a string to an integer) or produces concatenation instead of addition (in Java, 0 + "85" is the string "085"). Converting with int() or Integer.parseInt() before any arithmetic avoids both problems; float() and Double.parseDouble() convert decimal fields.

Finding the highest score without max()

The program top_score.py processes every record in scores.txt: it keeps a running total and a count to calculate the mean, and stores the highest score so far with the name that goes with it. It finds the maximum with a comparison inside the loop instead of the built-in max(), the version to use whenever built-in functions are not available.

The scores 72, 85, 64, 91 and 78 total 390, and $390 \div 5 = 78.0$. The Java version casts total to double before dividing; without the cast, int division would discard any fractional part of the mean.

Starting best_score at −1 works because no valid score is negative, so the first record always replaces it. A safer general method sets the best to the first record's values and compares the rest, which also works for data that may be negative. The total is an accumulator and count a counter, the two patterns behind most file statistics.

Lines that do not fit the format

Real files contain surprises: a blank line at the end, a missing field, a score typed as a word. A program that assumes every line is perfect crashes on the first bad one. int("sixty") raises a ValueError in Python, Integer.parseInt("sixty") throws a NumberFormatException in Java, and indexing fields[1] on a blank line raises an IndexError or ArrayIndexOutOfBoundsException, because splitting an empty string gives a single empty field.

Robust programs apply validation to each record before using it. They check that the line has the expected number of fields and catch the conversion exception for that line only, so that one bad record is skipped and counted instead of ending the whole run. The class gradebook program uses both checks. Whether skipping is acceptable depends on the purpose: a report can skip a corrupt line and say so, but a payroll program that silently skipped an employee would be worse than one that stopped with a clear error.

Writing to a text file

Writing lines in Python and Java

Writing reverses reading. The program opens the file in write or append mode, calls a write method once for each piece of text, and closes the file to flush the buffer. Python's write() and Java's FileWriter.write() both take a single string and add it at the file pointer. The program write_report.py writes a heading and one line per student from two parallel lists, then reads the file back to show what was saved.

The file report.txt now holds four lines, and running the program again replaces them with the same four lines, because write mode empties the file each time.

Newlines and numbers

Two details cause errors in many first file-writing programs. First, write() writes exactly the characters it is given and nothing more: unlike print() or System.out.println(), it never adds a newline. Records written without "\n" run together into a single line, as no_newline.py shows.

The file holds one line instead of two, and a program reading it later would split it into the fields Amara, 72Bao and 85.

Second, Python's write() accepts only strings. file.write(72) raises a TypeError, so numbers are converted with str() and joined with +, as in str(scores[i]). Java converts numbers automatically when a string is joined to them with + (names[i] + ": " + scores[i]), but FileWriter.write() called with a single int writes the character with that code, not the digits: file.write(72) writes the letter H. Building the whole line as a string first avoids both traps.

Appending to a text file

Append mode adds records without disturbing what is already there. The file pointer starts at the end of the file, so each write lands after the last existing character. If the file does not exist yet, it is created, which means the same code works on the first run and on every later run.

Log files

The most common use of appending is a log: a file to which a program adds one line each time something happens, and from which nothing is ever removed. Web servers, operating systems and apps all keep logs to record activity and to investigate faults afterwards. The Apache HTTP Server, for example, appends one line per request to its access log in the "Common Log Format": the client's address, the user, the time, the request, the status code and the size of the reply.

Source: Apache HTTP Server Version 2.4 documentation, "Log Files" (example entry).

Each entry is a record whose fields are separated by spaces, with quotes and brackets around fields that themselves contain spaces. A server that opened its log in write mode would destroy the history of every earlier request each time it restarted, which is why logs are always written in append mode. The program append_log.py adds a logout entry to a small log file of a school app and prints the result.

Before the program ran, log.txt held the first two entries; the third is the one just appended. Running the program a second time would add a fourth line identical to the third.

Appending relies on the file already ending with a newline. If the last line of an existing file has no newline (a file typed by hand often doesn't), the first appended record is joined onto it. Programs that write every record with a trailing "\n" never create this problem for the next program.

Closing files

What close() does

Closing a file ends the program's use of it. close() in Python and in each of Java's file classes does three things in order:

1. It flushes any data still waiting in the output buffer to the file, so everything written is saved.

2. It releases the file handle, telling the OS to remove the file from its table of open files and free the memory and buffers that went with it.

3. It lets other programs use the file. Some operating systems, Windows in particular, stop other programs from changing, renaming or deleting a file while it is open, and another program reading the file sees only what has been flushed.

After close(), the file handle can no longer be used: a further read or write raises an error (ValueError: I/O operation on closed file in Python; IOException: Stream closed from a Java BufferedReader or FileWriter, and IllegalStateException: Scanner closed from a Scanner). A file opened for reading should be closed too: no data is lost if it is left open, but the handle and its buffer stay allocated until the program ends, and a long-running program, such as a server, that opens files without closing them eventually reaches the OS limit on open files and can open no more.

Closing automatically: with and try-with-resources

Calling close() by hand has a weakness. If an exception is thrown between opening and closing, for example a ValueError while converting a bad score, the program jumps straight to the exception handler and the close() line never runs. One fix is to put close() in a finally block, which runs whether or not an exception occurred. Both languages offer a shorter and safer statement that does this automatically.

Python 2.5 added the with statement in September 2006 to replace the common try/finally pattern, and Java SE 7 added try-with-resources in July 2011. The program with_read.py reads one line inside a with block and checks the handle's closed attribute inside and after the block.

Inside the block the file is open; as soon as the indented block ends, Python has closed it, without any close() call in the code.

The table compares the three ways to make sure a file is closed. The last two are the same idea in different syntax, and both are the standard way to work with files in modern code.

In Java, a try-with-resources statement can open several resources at once, separated by semicolons, as the FilterCopy program did with its reader and writer; they are closed in reverse order. It may have catch and finally blocks of its own, or none, as in WithRead, where the throws IOException on main passes any exception on.

Handling file errors

What can go wrong

File operations depend on things outside the program: the file system, the device and other users. That makes them one of the most common sources of runtime errors in real programs, and the reason the Java compiler requires programs to handle them.

In Python, FileNotFoundError, PermissionError and IsADirectoryError are all kinds of OSError, so except OSError catches every file-system problem at once. In Java, FileNotFoundException is a kind of IOException. In both languages a handler for the specific exception must come before the handler for the general one, or the specific one is never reached.

Without a handler, the exception ends the program with an error message. The two-line program crash.py opens a mistyped file name.

Python stops at the first line with a traceback naming the exception (the file path is shortened here):

Catching file exceptions in Python

Exception handling lets the program respond to the problem instead of crashing: report it clearly, ask for another file name, or carry on with default data. Python's try/except statement wraps the operation that may fail; the optional else block runs only if no exception occurred, and finally runs in every case.

With the misspelt name, both versions print the same two lines; with the correct name scores.txt, they print the five records and then Finished.

Java has no else clause, so the code that uses the file goes inside the try block, after the line that opens it. That code is skipped automatically when opening fails, because the exception jumps straight to catch.

Checked exceptions in Java

Java divides exceptions into two groups. Most errors caused by faulty logic, such as NumberFormatException or ArrayIndexOutOfBoundsException, are unchecked: the compiler does not require them to be handled. File errors are different. IOException and FileNotFoundException are checked exceptions, and any method that can throw one must either catch it or declare it with throws in its header. A program that does neither does not compile.

Compiling NoCatch.java fails before the program can run, even though scores.txt exists:

There are two fixes. Adding throws FileNotFoundException (or throws IOException) to the header of main, as most of the short Java programs for reading and writing do, passes the exception up to the caller, and from main that means the program ends with an error message if the file is missing. Wrapping the code in try/catch, as MissingFile does, handles the problem inside the method. Short examples use throws to keep the code readable; finished programs catch the exception and tell the user what went wrong. Python has no checked exceptions: the programmer decides whether to handle file errors, and a forgotten handler shows up only when the error actually happens.

Putting it together: a class gradebook

A complete file-processing program usually combines every operation: it reads existing data, adds to it, calculates results and writes a report. The gradebook program does this for a teacher's file results.txt, which, like many hand-edited files, contains two lines that do not fit the format: a score written as a word and a blank line.

The program is decomposed into three subprograms, one per file operation, plus a main part that calls them inside one exception handler. add_result appends a new record, load_results reads every valid record into two lists and counts the lines it skips, and write_report overwrites report.txt with the number of students, the mean and the top student.

After Farid,88 is appended, the file holds seven lines. Five are valid (72, 85, 91, 78 and 88), and two are skipped: Chiara,sixty fails the conversion and the blank line splits into one field instead of two. The total is $72 + 85 + 91 + 78 + 88 = 414$, and $414 \div 5 = 82.8$.

Each design choice in the program applies a file-processing idea. The three modes appear once each: append to keep every past result, read to load them, write to replace the old report. Every file is closed by with or try-with-resources, so even a crash in load_results leaves no file open. The validation inside load_results turns two lines that would have crashed a simpler program into a count the teacher can act on. One limitation remains: the report counts Chiara's lost result only as "skipped", so a fuller version would also write the rejected lines to a separate file for checking.

Files, privacy and security

Text files are easy to read, which makes data easy to share but gives private data no protection of its own. Anyone who can open a file in a text editor can read every record in it: a gradebook, a list of email addresses or a file of passwords offers no protection beyond the operating system's file permissions. Personal data stored in plain text files has been exposed many times when a laptop was lost, a folder was shared too widely or a file was uploaded by mistake.

Programs that store personal data in files therefore apply several safeguards: they store only the fields they need, restrict who can open the file through the OS's access control, use encryption for sensitive data, and never write passwords themselves, only values derived from them that cannot be reversed. Log files raise the same issues: an access log that records every user's address and every page they visited is personal data, and keeping it longer than needed is a privacy risk as well as a storage cost.

Exam focus

File processing is examined on Paper 2, in the student's own language (Python or Java), and its one content statement uses the command term Construct. Expect a file to appear inside a larger programming question: a scenario gives the file's name and the format of its lines (often name,value), and the parts rise from short recall to writing code.

• State or identify (1–2 marks): the effect of a mode (write empties the file, append adds to the end), which line opens the file, what readline() returns at the end of the file.

• Explain (2–3 marks): why a file should be closed, why append mode is needed, why a value read from a file must be converted before arithmetic.

• Describe (2–3 marks): how an exception could occur when opening or reading a file and how it is handled.

• Construct (4–8 marks): read a file line by line and output a total, count, average, maximum or filtered list; copy selected records to a new file; append a validated record. Built-in functions such as max(), len() or sum() may be banned in such a part.

• Trace (2–4 marks): a reading loop, recording the line read, the variables and the output for a given file.

The Java example in part (b) is a fragment inside a main method declared throws IOException, with java.io.BufferedReader and java.io.FileReader imported. In an answer, imports and the class header are usually not required unless the question asks for a complete program, but the loop, the conversion and the end-of-file test always are.

Practice questions

1. State what happens to the existing contents of a file when it is opened in write mode. [1]

2. State the value returned by Python's readline() and by Java's BufferedReader.readLine() when the end of the file has been reached. [2]

3. Outline the purpose of a buffer when a program writes to a file. [2]

4. A file temps.txt contains one whole-number temperature per line. Construct code that outputs how many temperatures are below 0. [4]

5. Explain why a program that reads "85" from a file must convert it before adding it to a total. [2]

6. Distinguish between a relative and an absolute file path. [2]

7. The file notes.txt contains the four lines Files keep data safe, after a program ends., an empty line, and A text file holds lines of characters.. Construct code that outputs the number of lines, the number of blank lines and the number of words in the file. [5]

8. Explain one advantage of using with (Python) or try-with-resources (Java) instead of calling close() at the end of the code. [2]

9. A program appends a record to log.txt each time a user logs in. Suggest why append mode is used, and identify one privacy risk of keeping the log. [3]

10. Trace the Python loop line = f.readline() / while line != "": / n = n + len(line.strip()) / line = f.readline(), with n starting at 0, for a file containing the three lines ab, cde and f. Give line and n after each read. [3]

Summary

• Data in variables is lost when a program ends; files in secondary storage give persistent storage. A text file is a sequence of characters split into lines by newline characters (\n).

• Each line of a data file usually holds one record whose fields are separated by a delimiter such as a comma; text files are sequential, read from the start towards the end.

• Opening a file returns a file handle. Read mode ("r") fails if the file is missing; write mode ("w", new FileWriter(name)) empties or creates the file; append mode ("a", new FileWriter(name, true)) creates the file or adds to its end.

• Python reads with read() (whole file), readline() (one line, with its newline, "" at end of file), readlines() or for line in file. Java reads with Scanner (hasNextLine(), nextLine()) or BufferedReader (readLine(), null at end of file); both drop the newline.

• Every read moves the file pointer forward, so a second read() returns nothing; the end-of-file test must match the language's signal.

• Lines are parsed with split(","), strip() removes Python's newline, and numeric fields are converted with int() / Integer.parseInt() before arithmetic.

• write() adds no newline and Python's write() accepts only strings; every record written must end with "\n".

• Written data waits in a buffer until it is flushed; close() flushes the buffer, releases the file handle and lets other programs use the file. with and try-with-resources close files automatically, even after an exception.

• Missing files, permissions and device faults raise FileNotFoundError / OSError in Python and FileNotFoundException / IOException in Java; Java's file exceptions are checked, so they must be caught or declared with throws.

Self-check

• To do:

  I can explain why data in variables is lost when a program ends and how a file gives persistent storage.

• To do:

  I can describe a text file as a sequence of characters, and count the characters in a line including its newline.

• To do:

  I can identify the records, fields and delimiter in a data file, and explain why the delimiter must not appear inside a field.

• To do:

  I can state, for read, write and append mode, what happens to an existing file, where the file pointer starts and what happens if the file is missing.

• To do:

  I can write the Python open() call and the Java object creation for each mode, including new FileWriter(name, true).

• To do:

  I can read a whole file, one line, and every line until the end of the file in Python, and with Scanner and BufferedReader in Java.

• To do:

  I can state the end-of-file signal for readline(), readLine() and hasNextLine(), and explain why Java tests for null.

• To do:

  I can trace a reading loop, recording the line read, the file pointer, the variables and the output.

• To do:

  I can split a line into fields, remove the newline in Python and convert numeric fields before arithmetic.

• To do:

  I can find a total, count, mean and maximum from a file without built-in functions such as max() or sum().

• To do:

  I can write and append records, each ending with a newline, and copy selected records to a new file.

• To do:

  I can explain what a buffer is, what flushing does and why a file that is not closed may be empty.

• To do:

  I can use with and try-with-resources, and explain why they are safer than calling close() at the end.

• To do:

  I can describe how a file exception occurs and handle it with try/except or try/catch, and explain what a checked exception is.

• To do:

  I can validate the records in a file so that one bad line is skipped and counted instead of crashing the program.

1. (a)

   State the data type of total and of count.

   [2]2 marks

   Markscheme (a)

   total: decimal / float / double;
   count: integer;

2. (b)

   Identify three potential points of failure in this function/method.

   [3]3 marks

   Markscheme (b)

   Award [1] per point, up to [3 max].
   the file does not exist / cannot be opened (FileNotFoundError / FileNotFoundException);
   a line is not a valid number, e.g. ERR or a blank line (ValueError / NumberFormatException);
   the file is empty so count is 0 (division by zero: ZeroDivisionError in Python; NaN result in Java);
   the storage device fails while reading (IOException);
   the file is not closed if an exception happens part-way through;

3. (c)

   Describe the role of a finally block in this function/method.

   [2]2 marks

   Markscheme (c)

   Award [1] per point, up to [2 max].
   code in finally runs whether or not an exception occurs;
   it is used to close the file;
   so the file is not left open / locked / the resource is released even after an error;

4. (d)

   Construct, in Python or Java, an improved version that:

   • returns −1 if the file cannot be opened

   • skips any line that is not a valid number and outputs a message

   • returns −1 if there are no valid readings

   • always closes the file once it is open.

   [6]6 marks

   Markscheme (d)

   Award [1] per point, up to [6 max].
   try around opening the file;
   catches the file-not-found exception and returns −1;
   inner try around the conversion of each line;
   catches ValueError / NumberFormatException and outputs a message (without stopping the loop);
   count increased only for valid readings;
   closes the file in finally (or with / try-with-resources);
   checks count is 0 and returns −1;
   returns total / count otherwise;

   Note: Accept any structure that meets all four requirements.

   Example answer (Python):

   PythonCopy

   def average_reading(filename):
       try:
           file = open(filename, "r")
       except FileNotFoundError:
           return -1
       total = 0.0
       count = 0
       try:
           for line in file:
               try:
                   total = total + float(line)
                   count = count + 1
               except ValueError:
                   print("Skipped:", line.strip())
       finally:
           file.close()
       if count == 0:
            -
        total / count

5. (e)

   Explain why exception handling matters more for this program than for a program a person runs on a desktop computer.

   [2]2 marks

   Markscheme (e)

   Award [1] per point, up to [2 max].
   the sensor runs unattended, so no one is there to restart it after a crash;
   readings / data would be lost until someone notices;
   handling the error lets the program continue with the next reading / file;
   and record the problem (in a log) for later;

Example answer (Java):