Php Read a File Line by Line

How Can I Read a Large File Line by Line in PHP?

Last Updated — December 11, 2020

Every now and then, y'all will have to read and process a large file in PHP. If you are not careful when reading a large file, you lot will often finish up exhausting all the retentiveness of your server. However, in that location is no demand to worry considering PHP already provides a lot of inbuilt functions to read large files (by big I mean files with size over 1GB) either line by line or one chunk at a time. In this article, y'all volition learn almost the pros and cons of all these techniques.

On This Page

1. Reading a File Line by Line into an Assortment Using file()
2. Reading a Large File One Line at a Time Using fgets()
3. Reading a Big File in Smaller Pieces Using fread()
4. Quick Summary

Reading a File Line past Line into an Array Using file()

You can use the file() function in PHP to read an unabridged file into an array. This office stores each line along with the newline graphic symbol in its ain array element. If you intended to read a file line by line and shop the consequence in an assortment, the file() function seems to a natural selection.

One major drawback of this method is that it volition read the whole file at once. In other words, information technology solves the problem of reading a file one line at a time but it withal reads the whole file at once. This means that you lot won't exist able to utilize information technology to read very large files.

PHP

            $life_of_bee_lines = file('life-of-the-bee.txt'); $short_lines = 0;  // Output — Full Lines: 6305. echo "Total Lines: ".count($life_of_bee_lines).".\north";  foreach($life_of_bee_lines as $line) {   if(strlen($line) <= 30) {     $short_lines++;   } }  // Output — Full number of brusk lines: 778. echo "Full number of short lines: $short_lines.\due north";                      

In the instance higher up, I have read a text file chosen The Life of the Bee from Project Gutenberg. This file is but 360kb long and then reading it using the file() part was not a problem. Now we volition learn how to read a file that might be over 1GB in size without exhausting our retention.

Reading a Big File One Line at a Time Using fgets()

The fgets() part in PHP reads the current line from an open up file pointed to past the resource handle. The office stops reading after reaching a specified length, encountering a new line or reaching the terminate of file. It likewise returns FALSE if at that place is no more information to be read. This means that we can check if a file has been read completely by comparing the render value of fgets() to simulated.

If you lot take not specified a particular length that this function should read, it will read the whole line. In other words, the maximum retention that you need with fgets() depends on the longest line in the file. Unless the file that you are reading has very long lines, you won't frazzle your memory like you lot could with the file() function in previous section.

PHP

            $handle = fopen("life-of-the-bee.txt", "r"); $total_lines = 0; $short_lines = 0;  if ($handle) {     $line = fgets($handle);     while ($line !== false) {         $total_lines++;         if(strlen($line) <= 30) {           $short_lines++;         }         $line = fgets($handle);     }     fclose($handle); }  // Output — Total Lines: 6305. echo "Total Lines: $total_lines.\n";  // Output — Total number of short lines: 778. echo "Full number of short lines: $short_lines.\n";                      

Inside the if block, we read the beginning line of our file and if it is not strictly equal to false we enter the while loop and process the whole file one line at a time. Nosotros could besides combine the file reading and checking procedure in one line like I have washed the post-obit example.

PHP

            $handle = fopen("life-of-the-bee.txt", "r"); $total_lines = 0; $short_lines = 0;   if ($handle) {     while (($line = fgets($handle)) !== faux) {         $total_lines++;         if(strlen($line) <= thirty) {           $short_lines++;         }     }     fclose($handle); }  // Output — Total Lines: 6305. repeat "Total Lines: $total_lines.\northward";  // Output — Full number of short lines: 778. echo "Total number of short lines: $short_lines.\northward";                      

I would like to point out the just similar file(), the fgets() part reads the new line character every bit well. Y'all can verify this past using var_dump().

Reading a Large File in Smaller Pieces Using fread()

One drawback of reading files using fgets() is that you will need substantial corporeality of memory to read files which have very long lines. The situation is very unlikely to happen but if you lot can't make any assumptions about the file information technology is improve to read it in pocket-size sized chunks.

This is where the fread() part in PHP comes to our rescue. This function reads a file up to length number of bytes. Youare the one who gets to specify the length then you don't accept to worry about running out of retention.

PHP

            $handle = fopen("large-story.txt", "r"); $chunk_size = 1024*1024; $iterations = 0;   if ($handle) {     while (!feof($handle)) {         $iterations++;         $clamper = fread($handle, $chunk_size);     }     fclose($handle); }  // Output — Read the whole file in 13 iterations. repeat "Read the whole file in $iterations iterations.";                      

In the above case, nosotros read the big text file (1024*1024) 1MB at a time and it took thirteen iterations to read the whole file. You tin set up the value of chunk_size to a reasonable value and read a very large file without exhausting your retentiveness. You could use this technique to read a 1GB file using only 1MB of memory in about 1024 iterations. You can also read even larger files and increase the chunk_size if you want to keep the number of iterations low.

Quick Summary

Let's recap everything that we have covered in this tutorial.

1. If you desire to read files line by line into an array, using the file() role would be the right choice. The simply drawback if that this role volition read whole file at once and so you won't be able to read very large files without exhausting the memory.

2. If the files you are reading are very big and you lot have no plans to store the individual lines in an array, using the fgets() role would be the correct selection. The amount of memory that your script needs will just depend on the length of largest line in the file that y'all are reading. This means that the file could be over 1GB in size but if the individual lines are non very large, yous will never exhaust your memory.

3. If you are reading large files and take no idea how long individual lines could exist in that particular file, using fread() would be your best bet. This function lets you specify the size of chunks in which you want to read the file. Another advantage of fread() over fgets() is that you will not have to perform a lot of iterations if the file y'all are reading contains large number of empty lines.

Let me know if in that location is anything that you would similar me to analyze. As well, you are more welcome to annotate if yous know other techniques for reading large files one line at a time in PHP.

Rate this postal service —

Loading...

alexanderellow1964.blogspot.com

Source: https://tutorialio.com/read-a-large-file-line-by-line-in-php/

Share this post