Racing Software Inconsistencies
-
Sorry but going to have to start a new thread on this to give it more attention. (Previously posted something on another thread showing a bug or two)
So I have finally gotten around to fully spreadsheeting up all of my selections that BFBotmanager placed and comparing to the horse racing backtesting software and there are just so many inconsistencies I can see so far it makes the backtesting rather pointless (sorry to say this but there is no other way to put it, not trying to be rude).
I found a horse that got laid at 2.42 on the lay the 3rd fav strategy (3rd fav would never be that price in racing or any other market so how could that ever have been a selection? Maybe a rule 4 I guess? If that is the case this needs to be marked in the software. In fact I just checked the race in question and it was the clear fav the whole of the show so almost certainly no R4 - https://www.sportinglife.com/racing/racecards/2023-10-31/curragh/live-show/766417/sp-or-better-guaranteed-with-tote-irish-european-breeders-fund-maiden)
Then I started looking at the Back pace 8-10 strategy and trying to make head nor tail of the wildly different selections.
Here are the last few on the BTC software (This is set to bet 1 minute before the off to match my BFBM bot):
Now compare this to my bets:
First two match up... HOWEVER. The BTC strategy is set to:
So why are there two selections on this race, the Wolves 20:30
Same thing happened in the Lingfield 14:53, 2 selections backed.
(My BFBM bot "correctly" backed 2 selections as I forgot to set it to not do that)Apart from that there are 3 horses that appear on my bet list that do not appear on the backtesting software:
Muscika, Big Dream and Pulse Of Shanghai
There is also 1 bet that appears on BTC that does not appear on my list:
Havana Goldrush
To find so many discrepancies in such a small sample of the bets and on the very first few strategies I have bothered to check really does not fill me confidence in using it until this stuff is fixed.
@Adam I would really appreciate and respect your input here can you please let me know your thoughts? What you've built is bloody amazing for such a small dev team but there are clearly some issues that need fixing here.
Thanks
Andy
-
A Adam has marked this topic as solved on
-
Nice one Adam.
Yea I would be more of the thinking that if certain rules should not be selected at the same time it should just auto unselect them (maybe with a brief flashing animation of the unselected option, to alert user to what just happened)
Then it will be obvious that those two are incompatible (You could do this with the Max Qualifiers and Fav settings as well, same logic as we previously discussed, isn't it)
Cheers
-
@Dan-MacKinnon said in Racing Software Inconsistencies:
I know they’re getting the information from Betfair and if you ever wondered how reliable they are head over to the football forum on a busy match day!
This is true... you'd be amazed how many markets get marked in-play early, or just get completely deleted and recreated later. We've even seen several markets get assigned to the wrong country!Their market and/or order streams randomly drop regularly as well, leaving us with small gaps in the price history which we need to plug later.
We have a pretty decent set of audit scripts built now to catch these and other issues.
Aside from that, sometimes it's a case of people needing to think through exactly what they're expecting the software to do, but sometimes (as in this one), it's a case of the software not behaving intuitively.
The possible combinations of rules and options are so vast that there are sometimes use-cases we haven't considered. In this case I think we can change it to work more as expected
-
@Andrew-Gray yeah that’s fair but the BTC guys have always said it’s there as a tool not a magic formula. I think the user needs to have a bit of horse racing knowledge before entering and that’s up to the individual to do their research. Or ask on the horse racing forum. I had very limited knowledge but the guys over there helped and I’m sure I asked tons of stupid questions early on (still do no doubt!).
In theory there’s nothing to stop you creating a strategy where you back the horse that finished 1st in every race and you’ll get 100% strike rate.
I’m not saying the software is perfect but I’ve messed about and asked questions and now my results line up 99% of the time. The few that don’t I tend to give Adam the benefit of the doubt because I know they’re getting the information from Betfair and if you ever wondered how reliable they are head over to the football forum on a busy match day!
-
@Andrew-Gray There is a bit of a conflict in logic here. Probably best for us to disable the favourites and outsiders rules for now if someone selects BSP as the bet type (interestingly, we already had some handling for this in the code, so someone must have already considered this, however it was currently commented out).
I can actually see a use-case for what you're doing though, which would be looking at whichever runners are the top 3 just before the race and placing a BSP bet on them.
Unfortunately, this isn't what the software is doing. The favourites feature looks at which horses are the current fave, 2nd fave, etc at a particular timestamp by checking the price rank of the horse the last time the rank changed. So in this case, it should really look backwards from the race start time and see which horses were rank 1, 2 and 3 the last time all the price ranks were recalculated.
However, if someone selects the BSP bet type, it currently sets the lower limit of the price sample to the race start time, because you can't bet BSP before the race, and the software has ended up finding the first instance of price ranks 1, 2 and 3 being set AFTER the race start, then retrospectively applying the BSP as your entry price.
For example Havanazam steamed a bit after the race started and became the 3rd favourite during the race, so it appears in the results.
I'll see if we can accommodate the use-case described above somehow for BSP bets, otherwise we'll just disable those rules for BSP bets like we do for the entry time settings.
-
Appreciate all the comments and @Dan-MacKinnon - I agree it seems a bit pointless using this setting when you could just use say 1 second before the race and that should realistically match whatever BSP is in the large majority of cases, and I did that and the results look much more realistic.
I was just highlighting something that seemed odd when we do select BSP as an option and select 1,2,3 etc favourites.
e.g. why is there a 33/1 winner in the set when its v unlikely for that to be in top 3 favs.
I just checked the race and it was the 6th favourite, 3rd fav went off at 10/3 so it's not even close.
Obviously if you play around with software long enough and do your checks, you can just work this out but for someone new to it, they might select this option and naively think they've struck gold and start pumping money on it
so thanks to Adam for looking into it as always 
-
I don’t think BSP is the correct back setting for this strategy. If you think about how you would actually trade this strategy it would be nearly impossible to match the software because it’ll depend on when you place your trade regardless of BSP.
Instead, try it with entry price and set a wide range. Then enter whenever you think you or your bot would enter. I usually go 5 seconds before race and it matches up with the software 99% of the time.
-
@Andrew said in Racing Software Inconsistencies:
@Andrew-Gray Based on your screenshot you've excluded 1st, 2nd and 3rd favourites so any winners will have high odds.
He has them in the "include" dropdown.
I'll take a look at this now and clarify what's going on.
-
@Andrew-Gray the main issue is you are taking BSP which isn't known till after the race starts and excluding top 3 favourites.
For example: top 5 horses odds at BSP 2,4,6,8 & 10 you want to place bets at 6,8,10 but once BSP is known those odds might have changed to 2,4,6,6 & 20 so those bets wouldn't get matched. I had this moment when I was trying out a strategy you effectively need a time machine to place the bets correctly. -
@Andrew-Gray Based on your screenshot you've excluded 1st, 2nd and 3rd favourites so any winners will have high odds.
-
Just found something else a bit odd while playing around with the favourite settings.
Here is the filter file but below is hopefully an easy enough explanationSet Entry Price to "Take BSP"
Set Favourites to 1,2,3
Profit is ridiculous see screenshot:
Looking at individual bets section briefly there are things like 33/1 winners in there so it can't be filtering it as per the favourite instructions?
Is BSP entry price another thing that cannot be combined with Fav settings?
-
Apologies for the tardy response Adam
OK honestly I am not sure what the intended behaviour was because I didn't make this filter however.... I would guess that it should behave as such:
If there are more than one qualifier based on all the rules NOT relating to favourite position, it does not bet at all regardless of fav positions.
If there is only 1 qualifier based on all the rules NOT relating to favourite position:
It should then check fav position and only bet on that one qualifier if it's in the 1,2,3 fav position, therefore if 4th fav at 10 minutes becomes 3rd fav at say 5 minutes, the strategy would still bet on it.
Hope that makes sense.
I don't know how you would include this addition to the software and make it clear to users how the different options would affect the results, or even if that way of doing things would be useful to anyone/generate profit but that's just how I would have imagined it would work
-
@Andrew-Gray What I'm trying to get to is what you're trying to do "in real life".
You're trying to evaluate the 1st, 2nd and 3rd favourites in the last 10 minutes before the race, but ignore the race if more than one horse qualifies under your strategy rules.
This would mean if more than horse became the 1st, 2nd or 3rd favourite at any point during those 10 minutes (and met your other criteria), you would want to ignore the race.
This doesn't sound realistic as there's still an element of having to predict the future, so is it more like you're looking at which horses are the top 3 faves at exactly the 10 minute mark and if only 1 of those top 3 qualifies at that point, you're happy to place the bet, otherwise if it's more then you ignore the race?
If so, what would you do if none of the horses qualified at exactly the 10 minute mark, but after say 1 minute, the 3rd and 4th favourite swap and now the 3rd favourite qualifies? As it's the only horse that qualifies, is it a bet?
A basic summary of the above would be: "Bet on the first horse that meets my criteria in the last 10 minutes, as long as it's the only one at that time that qualifies".
My other question is how does BFBM actually handle this? We adjusted the favourite rule to try and match what they're doing, based on discussion in the Changelog thread. I was under the impression that their favourite setting completely overrode the max qualifiers setting, but maybe that's not correct.
-
OK yea that makes sense Greg thanks. Bit fiddly but I'll give it a go. Thanks!
-
Here is the original filter.
As you can see it has option for 1st 3 in the betting only:
And also max quals 1:
However this is currently set to bet 10 mins from the race start on the BSP 1,2,3 fav which obviously does not make sense as we cannot predict that in the future.
If I delete that rule, and set the 1,2,3 fav in the right hand side window so getting the favourite at time of bet element correct, it negates the only 1 qualifier rule.
I am assuming there is no way around this the way the software is currently set up
-
@Andrew-Gray said in Racing Software Inconsistencies:
- How can I backtest that then? Because the original filter used BSP favs and then the max qualifiers setting thereby successfully discarding any races with more than one BSP 1-3 fav that qualified. But we know we can't use the BSP fav option for successful backtesting because you don't know what they are until race time.
And if I use the fav at time of bet option, it overrides the max qualifiers, so it's a catch 22?
Presuming there is no way of doing this?
If you can explain exactly what you're trying to do with a couple of examples of what you would and wouldn't have bet on in some given situation, I can tell you what's possible. At the moment I'm not even clear on what you want to happen.
- Why not add something in so that if the fav option has been selected in the second section then user selects max quals, it automatically unselects the fav option?
Because the fave option is the more specific one and modifies the qualifiers limits, not the other way around. Effectively the fave option changes "max x qualifiers" to "first x qualifiers".
As I said, I will make this clearer in the software.
- How can I backtest that then? Because the original filter used BSP favs and then the max qualifiers setting thereby successfully discarding any races with more than one BSP 1-3 fav that qualified. But we know we can't use the BSP fav option for successful backtesting because you don't know what they are until race time.
-
@Andrew-Gray Apologies for butting in, but if I understand this right, you want to bet on either the 1st, 2nd or 3rd Fav (at the time of placing the bet), but only if the other two don't qualify? So for example bet on the Fav, but only if the 2nd and 3rd Fav's don't qualify. I assume that you have your BFBM bot set up to make this decision.
So, you set the software with this strategy, but ignore the "Max Qualifiers" bit, because as @Adam says this is incompatible. The software will pick all horse that qualify as either 1st 2nd or 3rd fav. If you group the results by "Favourite" and include "Favourite" in the Customise Your Results, select columns bit.
You will end up with a grouping of summary results by the Favourite at the time of bet. And the individual results will include which favourite position each selections was at at the time of bet, you could then export this to Excel (or whatever) and look at each Favourite Position "off-model" and remove races where the software "placed" more than one bet.
-
@Adam Yea I get (and already got) where you are coming from and I said I would use the ticket system in future, not really sure what more you want from me on that front.
Going back to the issue at hand with the favourites and max qualifiers not being compatible with each other:
-
How can I backtest that then? Because the original filter used BSP favs and then the max qualifiers setting thereby successfully discarding any races with more than one BSP 1-3 fav that qualified. But we know we can't use the BSP fav option for successful backtesting because you don't know what they are until race time.
And if I use the fav at time of bet option, it overrides the max qualifiers, so it's a catch 22?
Presuming there is no way of doing this? -
Why not add something in so that if the fav option has been selected in the second section then user selects max quals, it automatically unselects the fav option? So it becomes visually and instantly obvious to user that you cannot do both, without them having to trawl through all the change logs to work that out (or bug you about it
). Simple UI fix so no confusion in future? (I'm not sure how many of those other options in that section are mutually exclusive but you could do it for any that are as well)
Cheers
-
-
A Adam has marked this topic as unsolved on
-
@Andrew-Gray said in Racing Software Inconsistencies:
@Martin - I feel like I've highlighted an issue here that others may have found useful knowing about so I don't really feel you need to be too dissapointed about it
Appreciate that the support tickets are the easiest way to track these for you guys so will do that in future.It's not just for our benefit when tracking things. When I get up at 4am and see a post called "Racing Software Inconsistencies" which lists 6 or 7 different "issues" with sensationalist language like this...
Sorry but going to have to start a new thread on this to give it more attention.
there are just so many inconsistencies I can see so far it makes the backtesting rather pointless
so many discrepancies in such a small sample of the bets
there are clearly some issues that need fixing here
...I basically have to drop everything and spend the next 2 hours tracking a load of runners, prices etc to provide an explanation, because it obviously looks bad.
I don't mind tracking this data to provide that explanation. It would be very difficult for you to do it yourself, that's kind of the point of the software. But as you have seen for yourself in this case, the vast majority of the time, when people say something's broken, it actually turns out to be user error (although "error" is a bad term as I'm not saying they're doing anything wrong, just that there tends to be another explanation) and it's frustrating to have that stuff reported as some failure of the software.
Yes people need to know about the genuine issues, but it would be nice to have the chance to set you straight on what is and isn't an issue first, and to form a clear understanding of any actual issues, so that it isn't just a knee-jerk reaction to "ARGH EVERYTHING'S BROKEN!" when that's largely untrue.
Once we've identified what actually needs fixing, we announce that here anyway with a clear explanation of what's gone wrong and what we're doing about it, and when the fix is published, it goes in the public changelog and you get credited anyway.
You're a software engineer yourself Andy so I'd be very surprised if you didn't understand where I'm coming from on this.
Here's another quick one that may not be a bug so just a question: How soon before a match should I expect to see Betfair odds filtering in?
The next 24-36 hours of prices are shown. More info in the OP on the software support thread.
-
@Adam Cracking reply thanks Adam
@Martin - I feel like I've highlighted an issue here that others may have found useful knowing about so I don't really feel you need to be too dissapointed about it
Appreciate that the support tickets are the easiest way to track these for you guys so will do that in future.Here's another quick one that may not be a bug so just a question: How soon before a match should I expect to see Betfair odds filtering in?
I am looking at Premier League matches for Saturday and there are no Betfair odds in the filter software yet.
Thanks
