Curday.dat Data Format

[AriX]

I've been attempting to get to know the curday.dat channel data format this morning through "reverse engineering" (basically just trial-and-error hex editing :p), here's what I've found so far.

The curday.dat file, which contains listings for the current day, essentially has the following format:

HEADER
CHANNEL INFORMATION for channel X
PROGRAM INFORMATION for program X
CHANNEL INFORMATION for channel X
PROGRAM INFORMATION for program X
...and so on...
CHANNEL/PROGRAM INFORMATION for TV Guide copyright block (which is registered as channel PRV002)

I have not gotten into looking into the header too much yet, but it seems to have some information on the location of the TV Guide machine. In a hex editor, the ASCII side of the header looks like this:

Code: Select all

AE3366N..4YYNNYYNNl..0.DREV 5.KHHR.Hawthorne.91.91.131.1348

The first thing to note here is the word Hawthorne, meaning Hawthorne, CA, which is an area near Los Angeles where this machine seems to have been located. Interestingly, KHHR, which also shows up at the bottom of the screen in diagnostic mode, seems to be the code of Hawthorne's local airport. (?) I haven't done a ton of research into what the rest means, although the 91 may be of significance because 91 days after January 1st is April 1st (on non-leap years), and that is how the dates are stored in Prevue's listings system. Even so, changing the 91s to 92 does not seem to change anything. The date that curday.dat reflects seems to be stored in another file, but I haven't confirmed that yet.

One other cool thing I noticed is that typing shift-D in the guide does indeed reload the current listings, which is quite convenient for hacking around with curday.dat. More to follow.

[AriX]

On to the channel data format itself. A typical channel block looks like this on the ASCII side:

Code: Select all

[   2 ......CBS006.KCBS....................00...CBS006.

Obviously most of the actual attributes are stored as hex codes that can't be seen on the ASCII side, but let's focus on this first for a second. First, you see the open bracket, which just seems to mark the beginning of a channel block (and the end of the previous one). Then, there are a few spaces, followed by a channel number. On channels without a number (the pseudo-channels that are just messages, like "Effective February 1st, our Torrance & Hawthorne Service Center hours will be..."), the number is simply another space. After another space, there are a series of 0x00s, followed by the internal name of the channel, which is usually a six character code.

Since there are multiple (a lot, actually) of CBS affiliates/O&Os in the country, each one has its own unique code. In this case, CBS006 refers to KCBS, which is LA's CBS station. Some channels only have one version for the entire country, like TWC001, and some have two, like ETV001 and ETV002 (002 being E!'s west coast feed, which is presumably the same as ETV001 but 3 hours delayed).

The unique channel code is followed by an 0x00 and then the name of the channel that shows up in the listings. Some channels do not have a name, like the "Effective February 1st" notice at the beginning of the listings and the TV Guide copyright notice at the bottom. After the name, you see several dots, which are actually the channel's "flags". They seem to signify things like color (regular, highlight, or alt highlight), whether or not the channel is ditto (meaning the same listings continue at all times), whether or not the channel is stereo, whether it's a real channel or just a notice, and some extra stuff if the channel has Pay-Per-View information attached to it (indicating that the listings should pause over the channel and/or that there is more information about it after the listings scroll). Here is an example of these flags as they appear for L0708J (the "Effective February 1st notice", which is by default highlighted in red):

Code: Select all

00 00 00 00 00 00 00 00 00 A2 FF FF FF FF FF FF 00 00 00 00 00 00 82 FF FF

As you can see, first there's a bunch of padding for no apparent reason, and then a byte (in this case, A2) that sets several flags. Here are all of the different combinations of these flags I could find in curday.dat. (Thanks to swest77 for his table in another thread that made figuring out which channels have which flags much much easier!)
[html]<table>
<tr>
<th>Flags</th>
<th>Byte</th>
</tr>
<tr>
<td>None</td>
<td>0x01</td>
</tr>
<tr>
<td>HILITESRC CAF_PPVSRC</td>
<td>0x12</td>
</tr>
<tr>
<td>None HILITESRC CAF_PPVSRC</td>
<td>0x13</td>
</tr>
<tr>
<td>DITTO</td>
<td>0x20</td>
</tr>
<tr>
<td>None STEREO</td>
<td>0x81</td>
</tr>
<tr>
<td>HILITESRC SUMBYSRC CAF_PPVSRC STEREO</td>
<td>0x96</td>
</tr>
<tr>
<td>DITTO STEREO</td>
<td>0xA0</td>
</tr>
<tr>
<td>HILITESRC DITTO STEREO</td>
<td>0xA2</td>
</tr>
<tr>
<td>ALTHILITESRC STEREO</td>
<td>0xC0</td>
</tr>
<tr>
<td>DITTO ALTHILITESRC STEREO</td>
<td>0xE0</td>
</tr>
</table>[/html]
None - I don't quite understand this one. What is there none of? I don't get the pattern here.
DITTO - Seems to indicate that the listings do not change, or do not change often. May be responsible for the double arrows on either end of the listings?
CAF_PPVSRC - Marks the channel as having Pay-Per-View. Not 100% sure what that affects.
SUMBYSRC - Marks the channel as a summary source, meaning that after the listings are done rolling around, some of the programs that are going to be on the channel for that day will be listed on that grey background with a title of "Summary of <CHANNELNAME>".
STEREO - Marks the station as having stereo. Not sure why this is necessary.
HILITESRC - Red gradient highlight.
ALTHILITESRC - Blue gradient highlight.

After the flag byte, we have 6 bytes of 0xFF if the channel is to be displayed in the listings (or apparently sometimes something else - in the case of The Weather Channel, this is replaced with FC FF FF 03 00 00. I don't know what difference it makes), or 6 bytes of 0x00 if it should not be. Then, 6 more bytes of 0x00 for no discernible reason, and then either 0x82, 0x8A, or 0x8B (not sure what this affects yet), and FF FF 30 30 00 00 03, the internal name of the channel again, another 00, and finally the program data. Here's a full block of the TV Guide Channel on channel 12 in both ASCII and Hex just for reference.

Code: Select all

[   12 ......TVGC...TVGC....................00...TVGC.

Code: Select all

5B 20 20 31 32 20 00 00 00 00 00 00 54 56 47 43 00 00 00 54 56 47 43 00 00 00 00 A0 FF FF FF FF FF FF 00 00 00 00 00 00 82 FF FF 30 30 00 00 03 54 56 47 43

[AriX]

Here's the table of the various bitflags that are bitwised together to get the byte that sets several of the channel attributes:
[html]<table>
<tr>
<th>Flag</th>
<th>Byte</th>
</tr>
<tr>
<td>None</td>
<td>0x01</td>
</tr>
<tr>
<td>HILITESRC</td>
<td>0x02</td>
</tr>
<tr>
<td>SUMBYSRC</td>
<td>0x04</td>
</tr>
<tr>
<td>VIDEO_TAG_DISABLE</td>
<td>0x08</td>
</tr>
<tr>
<td>CAF_PPVSRC</td>
<td>0x10</td>
</tr>
<tr>
<td>DITTO</td>
<td>0x20</td>
</tr>
<tr>
<td>ALTHILITESRC</td>
<td>0x40</td>
</tr>
<tr>
<td>STEREO</td>
<td>0x80</td>
</tr>
</table>[/html]

VIDEO_TAG_DISABLE was not in curday.dat, but when I use the value 0x09 as a flag for a channel and use the H listings diagnostic, the flags show up as "None VIDEO_TAG_DISABLE". When it's applied, the channel doesn't show up in the listings at all. However, when it is bitwised with ALTHILITESRC, it does show up.

[AriX]

YES! I was finally able to change the date of the listings in curday.dat!
All that's necessary is to change the 91.91 in the header of curday.dat to another day expressed in how many days it's been since January 1st and then do the same in the header of nxtday.dat (before it said 92.91 in nxtday.dat, I made it say 93.92 to increment it by a day).
Here's a screenshot showing listings on April 2nd:

[Bolt96]

Wow cool! Good progress so far!

[AriX]

Wow cool! Good progress so far!

Thanks!
Anyways, I'm gonna get some real work done for a bit, and maybe I'll tackle program listings tomorrow.

[AriX]

On to program blocks!

Program blocks appear to be pretty simple, but I do not understand most of their attributes as they do not appear to have a visible effect. Here's an example program block:

Code: Select all

.32.1.22.0.0.CBS News . |.

The first dot is an 0x00, which just serves as a divider between programs. Then comes the timeslot. Each timeslot is a half-hour block, so there are 48 timeslots in a day - however, since the timeslots start at 1 instead of 0 they go from 1 to 49. Also, they don't start until 3:00AM. For example, timeslot 27 would resolve to 4:00PM. In this case, we have timeslot 32, which means 6:30PM. Then we have 4 fields that I unfortunately don't understand - changing the values does not create any noticeable difference in the guide. 0x00s serve as dividers for the fields. Then, we have the name/description of the program followed by any symbols desired for the program. For example, if the program uses closed captioning, an 0x7C may be used to produce a closed captioning symbol. Any number of these symbols may be used; they are just characters in the PrevueC.font. I put a little more information on these symbols over here. In this case, two symbols are used: the headphones symbol (which shows up on the ascii side as " .") and the closed captioning symbol (which shows up on the ascii side as " |"). Finally, there's another 00 as a divider.

One thing to note about program blocks is that they do not contain information on when the program ends - the program simply ends when the next one begins.

[nwgatwcfan]

Hi there. I am a new member to the site. I figured a few extra things out on this section of Curday.dat

AE3366N..4YYNNYYNNl..0.DREV 5.KHHR.Hawthorne.91.91.131.1348


The AE3366N is the settings in Diagnostic mode -> A E 3 36 6 N

The 4 is the Time Zone setting in the second section
The first Y is for DST
The second Y is for Cont.
the rest of this block is for ad settings.

The KHHR is indeed the Airport in Hawthorne CA. If you change this setting, the new setting is listed in diagnostic mode. I know why this is here. In the black version of the guide, there was a setting for Prevue Weather that would come up in the scroll every half-hour or so. When they switched to the blue grid, they carried this over as an additional feature. When they switched to the newer Prevue logo and the Prevue News and Weather, it was displayed in the top portion of the Guide with the weather icons and temperatures, etc. The KHHR is also where Weather information is gathered in Hawthorne. So the guide must be pulling up the weather for KHHR Airport and naming it Hawthorne for the city in the Prevue Weather listings.

I'm not sure if they were sending weather data directly through the same feed as the listings data, but I'm pretty sure that it came from the same data stream.

Hope this helps a bit in your quest.

Steven

[AriX]

Hi there. I am a new member to the site. I figured a few extra things out on this section of Curday.dat

Awesome! I feel silly for not seeing some of that earlier... Thanks! I hadn't even thought of the KHHR representing the weather source, but that makes sense now :p

[JoeTheDragon]

On to the channel data format itself. A typical channel block looks like this on the ASCII side:

Code: Select all

[   2 ......CBS006.KCBS....................00...CBS006.

[/code]

what hex edit app are you useing to get the text?