Fixing problems with installing Ubuntu from a USB flash drive

David W. Pierce  /   Scripps Institution of Oceanography /  28 May 2013

Description of the problem:

   Installing from the USB flash drive seems to work, but soon into the process you get this error message:

There was a problem reading data from the CD-ROM. Please make sure it is in the drive. If retrying does not work, you should check the integrity of your CD-ROM.

Failed to copy file from CD-ROM. Retry?
   
Retrying does not fix the problem. Using a new, or different, USB flash drive does not fix the problem. As far as you can tell, nothing fixes the problem. Screaming and cursing ensue.

How to fix the problem:

  1. Make your USB flash drive as per normal (i.e., the one that doesn't work). I use Linux Live USB creator
  2. If you are making the flash drive on MS Windows:
  3. If you are making the flash drive on Linux:

What does the script do?

I have never gotten Ubuntu on a USB stick to work. I know, I've read tons of comments
on the internet that people generally get this working, so why does it always fail
for me?

The crux of the issue seems to be that when the USB stick is written from an ISO image, the
filenames are written "as if" it were a CD-ROM. Now a CD-ROM has limitations as to the file
names that can be accomodated. The traditional explanation is that it can only be 64 characters.
On the other hand, many of the package names in Ubuntu LTS server (which is what I use)
have more characters than that in the filename.

So here is what I think is happening. I believe that most ISO Linux images use
64-char (max) filenames when they are created, in the belief that they will be written
to a CD-ROM. Perhaps when a *real* CD-ROM file system is given a file name that
is too long, the CD-ROM driver truncates that long name to a 64-char name. In that
event everything will work -- the filename was written to the CD-ROM with only 64 chars
max, and the CD-ROM driver, when given an overly long file name to read, will automatically
truncate the filename to 64 chars, and so finds the file under the name it was written.
However I speculate that the USB driver does not do this truncation when reading, since a
USB stick (as far as I know) has no limitations on the file name length. So, the USB driver is
handed a long file name to read, the driver does *not* truncate the file name, and the file
(which was written with a truncated name in anticipation of being written to a CD-ROM)
is not found. Errors result.

This script fixes the problem (at least, for Ubuntu LTS server 12.04.2, which is what
I tested it on). It renames the truncated files back to their original, un-truncated
names. You might well ask, if the files are written onto the USB stick in the first
place in truncated form, how can you possibly know what the original name of the file
was? Some characters were lost when the filename was truncated. Well, we can get the
original names of the files from the md5sum.txt file, which has a md5 checksum of every
file on the USB key (I think). The algorithm is to go through each filename in the md5
checksum file; if the file already exists under that name, then we are fine. If the file
is missing, then we check to see if a file exists with the truncated version of the
filename. If *that* file exists, then we rename the truncated file back to the original
filename.

Obviously this fails with very long file names that differ after 64 characters. For
example, if you try to write these two filenames to a CD-ROM they will overwrite
each other:

     package_with_very_long_filename_that_is_more_than_64_chars_long_version-0001.deb
     package_with_very_long_filename_that_is_more_than_64_chars_long_version-0002.deb

In that case this script will fail, but there is nothing I can do about that. The
USB key (and a CD-ROM version of it) would both be invalid in that case.

Acknowledgements

Thanks to Christopher Hinkle for fixing a problem with the windows version.

Licensing, etc.

The good thing about a perl script is that you can look through it with any text editor and make sure it's not doing something nasty. Please feel free to do so. And, if you come up with any improvements, I'd appreciate it if you emailed them back to me so I can add them in.

This is free and unencumbered software released into the public domain.
 
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
 
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
 
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to <http://unlicense.org/>


David W. Pierce
Division of Climate, Atmospheric Sciences, and Physical Oceanography
Scripps Institution of Oceanography, La Jolla, CA
dpierce@ucsd.edu