Cannot tune to some channels on linux (hdhomerun)

[sigl]

Hello, I retry using "allow_unicode_characters_in_generated_filenames" and the debug to ON.

The original program name is : "Le plus grand musée du monde"
The saved file created by SageTv is : "Le plus grand musée du monde - Le Petit Palais - 214054-0.mpg"




Finally the recording does not stay at 0 KB (i'm sorry), it grows correctly and the file can be read with VCL media player.

But the problem stay. After the recording end it create and error and the recording is not available in Sage Recordings but it exists in the recording directory.

Error Message Details

The device input "HDHomeRun 103958fe Tuner 1 Digital TV Tuner" failed to record "Le plus grand mus?e du monde" on CFTUDT on physical channel 29-29-1 at Thu 11/3 8:00. The TV source for this input may have become disconnected, powered off, has lost its signal, the channel is no longer available, or the device is failing.

I have include the sagetv_0.txt to help debugging


----
Update .....


My SageTV server is running on a Linux machine but the recording are saved on a mounted windows share (using /etc/fstab mount). I was wondering if the problem could be cause by this. So I remove the mount to make sure the recordings stay on Linux.. Now when the recording start it create two files !

The original program was "Ça finit bien la semaine"
The first filename create is : "?a finit bien la semaine - S07E06 - 213859-0.mpg" and it's 0 KB
the second filename create is : "Ça finit bien la semaine - S07E06 - 213859-0.mpg" and it size is correct.

But I got the same error as before.

[Narflex]

This is some kind of Java/platform issue...here's a very relevant part of the logs:

Thu 11/3 8:20:22.743 [AsyncWatch@29f7bdb3] Seeker.startRecord(HDHomeRun 103958fe Tuner 1 A[214054,212051,"Le plus grand mus?e du monde",74875@1103.08:00,30,T], currTime=Thu 11/3 8:20:22.742) currRecord=null switch=false
Thu 11/3 8:20:22.743 [AsyncWatch@29f7bdb3] Setting up MMC video for recording new show & tuning channel conn=HDHomeRun 103958fe Tuner 1 Digital TV Tuner
Thu 11/3 8:20:22.743 [AsyncWatch@29f7bdb3] Using quality setting "Great" for recording
Thu 11/3 8:20:22.743 [AsyncWatch@29f7bdb3] VideoStorage for new file: /var/media/tv - Leave Free 25.0 GB
Thu 11/3 8:20:22.745 [AsyncWatch@29f7bdb3] MediaFile unable to create file for next segment:java.io.IOException: No such file or directory
Thu 11/3 8:20:22.745 [AsyncWatch@29f7bdb3] Added:MediaFile[id=236812 A[214054,212051,"Le plus grand mus?e du monde",74875@1103.08:00,30,T] mask=TV host=SageTVSrv encodedBy=HDHomeRun 103958fe Tuner 1 CFTUDT format=MPEG2-PS 0:00:00 0 kbps [] /var/media/tv/Le plus grand mus?e du monde - Le Petit Palais - 214054-0.mpg, Seg0[Thu 11/3 8:20:22.743-Wed 12/31 19:00:00.000]]

Notice where it failed trying to create the file...it's because something went bad when SageTV was telling Java simply to create the file with the unicode filename.

But then when it asks the HDHomeRun code to record it, that works fine because the native code doesn't have an issue with the unicode filenames apparently...but Java isn't able to properly reconcile them.

I did some quick research on this...and it's very complicated, so I'm not sure exactly what to do in order to resolve this.

[sigl]

Ok if we forgot to use the Unicode way I think the problem could be solve for accents characters by removing them like mentioned in the following link:

http://stackoverflow.com/questions/3...-regular-lette


I think it would be easy to do that in the function createValidFilename in the MediaFile.java file something like this:

public static String createValidFilename(String tryMe)
  {
    if (tryMe == null) return "null";
    int len = tryMe.length();
    StringBuffer sb = new StringBuffer(len);
	
=====> tryMe= Normalizer.normalize(tryMe, Normalizer.Form.NFD);
=====> tryMe = tryMe.replaceAll("[^\\p{ASCII}]", "");
	
    for (int i = 0; i < len; i++)
    {
      char c = tryMe.charAt(i);
      // Stick with ASCII to prevent issues with the filesystem names unless a custom property is set
      if (Sage.getBoolean("allow_unicode_characters_in_generated_filenames", false))
      {
        if (Character.isLetterOrDigit(c)
          || (Sage.getBoolean("extended_filenames", false) && LEGAL_FILE_NAME_CHARACTERS.contains(String.valueOf(c))))
            sb.append(c);
      } else if ((c >= 'a' && c <= 'z') ||
        (c >= '0' && c <= '9') ||
        (c >= 'A' && c <= 'Z') ||
        // Jeff Harrison - 09/10/2016
     	  // Keep spaces and other extra characters in filenames
        (Sage.getBoolean("extended_filenames", false) && LEGAL_FILE_NAME_CHARACTERS.contains(String.valueOf(c))))
          sb.append(c);
    }
    return sb.toString();
  }

This way the accents would be removed but not then entire caracter it would create more realist filename.

[Narflex]

Do you want to do the change yourself? I'd recommend just doing the normalize if allowing unicode characters is not enabled (so you don't strip them for those who can handle them on their platform) and then using the rest of the code as is. I wouldn't do the replaceAll for non-ascii characters since that's already handled in the code below.

[sigl]

I would prefer if you do it. Here is the correct tested code. Don't forget to add the import java.text.Normalizer in the beginning of the file

public static String createValidFilename(String tryMe)
  {
    if (tryMe == null) return "null";
	
    tryMe= Normalizer.normalize(tryMe, Normalizer.Form.NFD);

    int len = tryMe.length();
    StringBuffer sb = new StringBuffer(len);

	
    for (int i = 0; i < len; i++)
    {
      char c = tryMe.charAt(i);
      // Stick with ASCII to prevent issues with the filesystem names unless a custom property is set
      if (Sage.getBoolean("allow_unicode_characters_in_generated_filenames", false))
      {
        if (Character.isLetterOrDigit(c)
          || (Sage.getBoolean("extended_filenames", false) && LEGAL_FILE_NAME_CHARACTERS.contains(String.valueOf(c))))
            sb.append(c);
      } else if ((c >= 'a' && c <= 'z') ||
        (c >= '0' && c <= '9') ||
        (c >= 'A' && c <= 'Z') ||
        // Jeff Harrison - 09/10/2016
     	  // Keep spaces and other extra characters in filenames
        (Sage.getBoolean("extended_filenames", false) && LEGAL_FILE_NAME_CHARACTERS.contains(String.valueOf(c))))
          sb.append(c);
    }
    return sb.toString();
  }

[Narflex]

OK, done.

[sigl]

Hello,


I think like the way you made the modifications there will be a problem with the for loop after. The tryMe result after the normalize function will not be the same size. It will grow as added characters will appears. I my mind we have to recalculate the size of the lenvariable after the normalize function.

[Narflex]

Originally Posted by sigl

Hello,


I think like the way you made the modifications there will be a problem with the for loop after. The tryMe result after the normalize function will not be the same size. It will grow as added characters will appears. I my mind we have to recalculate the size of the lenvariable after the normalize function.

Oh whoops...good point. I also updated the test cases so that it would catch this problem as well. Change committed.