Note that FileNotFoundError is preferred in modern Python instead of OSError. In case of old Python, one should narrow the catching to cover only the wanted case:

    import os, errno

    try:
        os.remove(temp_file)
    except OSError as e:
        if e.errno != errno.ENOENT:
            raise