Now I'm Lost!

The good news is the bot design is now under way. I started off today designing a routine that, with the click of a mouse, the day's data file, containing the results from the previous day, would be downloaded from the Betfair site, some of the data contained therein modified, then the file saved to my server before being fed into my database.

That worked I treat and I went on to modify it further so that it would do the same on any number of files en masse. That worked even better than I imagined downloading all 767 available win market files dating back to 2nd Sept 2008 in under 5 mins. 287 seconds to be precise.

Now all I need to do is scalp the Betfair website for Event ID and Selection ID in order to establish which horses are running on the day so that I can look up their past performance history. And that is where I'm lost. I cannot figure out what I'm trying to scalp for! I need the source of the menu that lists the events, and then from there the runners. I just cannot see how to access it at the moment.

While I'm sorting that out, I'm going to stop pre-race trading and concentrate on the laying. I'm continually throwing money away trying to trade pre-race which is compounded by the on-going software issues I'm having, which I won't go into here.

As you can see from the running total in the sidebar, the lay system has had a great few days with it touching 50 points for the month already. I am still having to manually enter the bets, but that won't change until I get the bot fully automated through the free API. That is something that isn't going to happen for a while as there's lots to get my head around before that.

The point is, it doesn't make sense to me to keep bashing away at the pre-race trading when I'm continually losing. A change is probably as good as a rest as they say.


The Geek said...


If you are scraping the website, try a tool such as URL snooper. Should point you where you need to go.

Alistair said...

Geek, is that you trying to fob me off with Windoze software again? LOL!

The actual scraping part I can handle, once I know what to scrape. The JSON filled Betfair website is confusing me. I can't make head nor tail of it at the moment. Indeed, I hadn't heard of JSON until I started investigating this a few weeks a go.

I'm wondering if it would be more straight forward to scrape the Betfair Lite site instead.

Anyway, thanks for the suggestion. It might take this old codger a bit longer but I'm sure I'll get there in the end.

Millionaire Fund said...

Sometimes a mix of the API and scraping is the way forward due to the restrictions on the free API. Like most things with the API there's rarely a call that does exactly what we want so it's a case of filtering the returned data to pick out the markets you need. Not too sure why the JSON side of things is confusing you Al as basically it's just data in a regular format. As the site pages are rendered with javascipt only part of the pages are in JSON format and it's just a case of stripping that out then encoding with php etc. I'll have a chat later see how you got on with the snippets I gave you.

Millionaire Fund said...

Heres some example to show the JSON Al

";//// just some breaks on screen to see the various stages easier

$fd = str_replace("while(1) {};", "", $fd);/// we need to strip to the json data only so replace the while(1) {}; which is used in the javascript

var_dump(json_decode($fd)); /////////lets dump the data

echo "

";//// just some breaks on screen to see the various stages easier

////lets dump the runners and selectionID


echo "

";//// just some breaks on screen to see the various stages easier



Alistair said...

Thanks for that MF... and the code snippets.

You'll have noticed from my post yesterday that I will be revamping my Linux server in the next couple of days. I'll get back to the coding then.