Hawkeye. Not infallible.

[Maklike Tier]

I'm starting to develop an issue with the Hawkeye system.

While I definitely agree that it's better than a line judge, I think there are occasions where it's just plain wrong, and it's really obvious how wrong it is when you're shown high-speed camera footage of the same point, versus the infographic.

Here's a case-in-point from the Lin Dan vs Chen Long final recently:



Discuss.

 

[penguinarchon]

Both the animation and high speed footage agree that the shuttle landed out

 

[stradrider]

I have seen this too and to me it's an argument against the idea that IRS is totally automatic, as some people claim.

The video replay shows max 1 cm out and in the animation it looks more like 10 cm. It is ok if it was an animation selected by a person, as the error is in the animation itself, and "out" is "out" no mater the animation. If the animation was done by censor reading, that's not good because it would have shown out even if the shuttle was just 1 cm inside the line... Which is not good at all...

 

[juneau-AK]

A similar situation was already brought to the attention of this forum - check http://bit.ly/2pBwy7M

Hawk-Eye has most likely made the outcome of challenge of line-call decision fairer, its tactical use by player notwithstanding. It certainly has helped umpires adjudicate fairly to potential line-judge bias. However, as the badminton operations fellow at the company states - tracking a shuttle is much harder than tracking a tennis ball due to the nature of a shuttle's flight path.

Historically, BWF approved the Hawk-Eye technology for first use at the 2014 India Open; at that time, there were no graphics (right BC-posters?), only the instant-review portion was available to players, just as it was available at the 2013 Malaysian Open SSF. It has since been a mainstay of the sport at most GPG and higher events. It is important to realise, no matter how accepted it is by players, and the popularity with spectators, that the technology is probability based, a predictive system. For instance, the second screen-grab is the actual picture, in technology parlance, it was the first measurement, the green-screen graphics is the result of a 'secondary measurement' that Hawk-Eye provided.

So what is being conveyed on the large screen inside the arena, and in the videos is the uncertainty and indeterminacy. To date, I have found no review of the number of challenges that were correct/incorrect, or the review of its accuracy. In tennis, the system was accurate to 3.6 mm (when introduced).

As the secretary general of BWF has been reported that they have engaged much in using new technologies to create added value for viewers and spectators. BWF ostensibly accepted the brand-reputation and global recognition among outer sports. Since its first introduction as instant review (for players ony), it has since added graphics (for large-screen viewing inside arena), and since 2015 has the fastest smash tracker available (not the same i-r cameras). The IPL (India, 2016) now has the flight-path of rallies shown, and I also recall the foot-tracing of the player; this latter could be some other technology. Hawk-eye uses SMART (for Synchronised Multi-Angle Replay Technology) now, and all this is great to enhance and optimise spectator and viewing experience.

Hawk-Eye in cricket and tennis is using same technology in two very different ways. If it was me, I would use the data that is gathered from the use to create statistics, explain tactics of player/s, and explain the recreated flight of shuttle-path properly. There is claim made by Hawk-Eye Innovations to help officiate service faults. Let us wish that first there is improvement in accuracy in line calls, both with use of technology and training of technical officials.

 

[2wheels04]

I'm starting to develop an issue with the Hawkeye system.

While I definitely agree that it's better than a line judge, I think there are occasions where it's just plain wrong, and it's really obvious how wrong it is when you're shown high-speed camera footage of the same point, versus the infographic.

Here's a case-in-point from the Lin Dan vs Chen Long final recently:

View attachment 183458 View attachment 183459

Discuss.

Very likely this is a human error, such as one where the technician loads the wrong rally for review.
It has happened before (tennis anyone?), and circles back to training. The organisers need people with proper training. no amount of high tech equipment can adequately correct that.

Just imagine, tomorrow, we may be watching the rally from the player's perspective, where there could be more than the 10 cameras with 5k resolution (as is with the current setup), and then the shuttle is shown to fall like this. Frankly, I do not want to suppose such scene.

 

[Maklike Tier]

My main issue is that the entire system is being touted as an infallible system. People have it in their minds that it's incredibly accurate and faultless. Remember those line calls that are called 'in', and the graphic zooms in - sometimes multiple times - showing the shuttle clipping the lines by millimeters? This is complete rubbish. What they're showing is a lie.

There's also two other points.

Firstly, That the cork on the end of the shuttle is a dome, so it's level of deformation is entirely speed and angle dependant. But what do we get via Hawkeye? A perfectly round dot of the same size, every time. Regardless of how the shuttle hits the ground.

Secondly, and I know this has been discussed before, but I have a hunch that it might actually be possible for the shuttles feathers to hit the ground at the same time if not milliseconds earlier than the cork. I haven't got a shuttle handy, but I suspect it just might be possible if the angle it's struck at is the same as that of the angle of the feathers. If so, whats the rules on that? At the moment Hawkeye places a round dot indicating whether it's in or not, but that is patently obvious not what happens.

 

[SSSSNT]

Well there's no faultless system. Anyone who believes that must not be very smart. It's true though that Hawkeye is incredibly accurate. It doesn't matter if it's wrong a few times out of thousands. After all, we accepted human referees and line judge whose margin of errors are much much higher than Hawkeye. Systems are always improved, perhaps in the future we'll have ultra tiny sensors around the court as a Hawkeye replacement. But for now, Hawkeye as the final say is fine.

 

[wahchai305]

from my limited knowledge - the IRS for badminton is based solely on high definition camera shots; there is NO prediction from hawkeye or any predictive system. the final animated picture showing whether the disputed line call is just that - an animation. there is no correlation of close is the line call.
as a player and as an official, on court or court side, a person do have a sense of whether the call is good/bad. more often then not, a challenge is made in hope, to buy time to break up the opponent's rhythm, or "what the heck, give it a try anyway".

 

[Maklike Tier]

Well there's no faultless system. Anyone who believes that must not be very smart.

I don't share your faith in humanity. Honestly, I think if you ran a poll, most people would think that the Hawkeye graphic represents what actually happened, when in reality it's an operators interpretation of what most probably happened.

It's true though that Hawkeye is incredibly accurate. It doesn't matter if it's wrong a few times out of thousands.

Except the next time the above example I quoted happens on championship point.

After all, we accepted human referees and line judge whose margin of errors are much much higher than Hawkeye.
Systems are always improved, perhaps in the future we'll have ultra tiny sensors around the court as a Hawkeye replacement. But for now, Hawkeye as the final say is fine.

And also just as we (Well, BWF, after much cajoling) implemented Hawkeye after far, far too many inaccurate line calls, if not constantly improved, so too will Hawkeye share the same fate. If you believe that standards are there to evolve and improve, you'd have to agree that at some stake, Hawkeye will also be deemed unacceptable.

 

[SSSSNT]

I don't share your faith in humanity. Honestly, I think if you ran a poll, most people would think that the Hawkeye graphic represents what actually happened, when in reality it's an operators interpretation of what most probably happened.



Except the next time the above example I quoted happens on championship point.



And also just as we (Well, BWF, after much cajoling) implemented Hawkeye after far, far too many inaccurate line calls, if not constantly improved, so too will Hawkeye share the same fate. If you believe that standards are there to evolve and improve, you'd have to agree that at some stake, Hawkeye will also be deemed unacceptable.

Well those are people who doesn't understand Hawkeye. But even them should believe there's no faultless system. If this is your problem then the solution is to educate people about Hawkeye so they understand how it works.

Still doesn't matter whether it's on the first point or championship point. Even the first picture shows that it is highly accurate.

I think this is making too big a deal on a fine system. Acting as if Hawkeye is some abomination. It is in fact, very highly accurate. Yes we should constantly make improvement. But we shouldn't overreact.

 

[visor]

We should understand that as with all science and technology, there's some confidence interval and error margin in accuracy. In tennis, Hawkeye is around 2-3 mm. In badminton, likely same.

As for the current Hawkeye system, just ask any pro players to see what they think, and I'm pretty sure all of them would never go back to the old days without it.

Sent from my SM-G930W8 using Tapatalk

 

[juneau-AK]

Well those are people who doesn't understand Hawkeye. But even them should believe there's no faultless system. If this is your problem then the solution is to educate people about Hawkeye so they understand how it works.

Still doesn't matter whether it's on the first point or championship point. Even the first picture shows that it is highly accurate.

I think this is making too big a deal on a fine system. Acting as if Hawkeye is some abomination. It is in fact, very highly accurate. Yes we should constantly make improvement. But we shouldn't overreact.

Several years ago, ITF (tennis) had adopted the technology, and many would remember the fiasco that had happened during the Mens match (Federer-Berdych, if I recall) - there was some shadow over the spot where the ball hit the court, and Hawk-Eye did not determine where.

Well, anyway, my scouts assure me, the chief of sci&tech at ITF had been reported (2005) that the technology operated within acceptable standards (there was some chatter in news-reports that it is 3.6 mm), and that the system's reconstructions were only a best guess of what happened and that to fully explain such concepts would only confuse the public.

 

[Maklike Tier]

We should understand that as with all science and technology, there's some confidence interval and error margin in accuracy. In tennis, Hawkeye is around 2-3 mm. In badminton, likely same.

Did you look at the images on my original post? It would be lucky to be 2-3 centimeters. In fact, with that example above - and it wasn't an isolated incident from the Asian Championships - the high speed photo you could almost argue the shuttle may have clipped the line, but in the amimation, it's 7-8cm out.

As for the current Hawkeye system, just ask any pro players to see what they think, and I'm pretty sure all of them would never go back to the old days without it.

Strawman argument. I never suggested that the old system was better.

The point of my post is, is that the system needs scrutiny - just as the old system before it - because without it, we'll be lulled into thinking it's an amazing, infallible system which it clearly isnt.

So here I am, giving it scrutiny

 

[juneau-AK]

I don't share your faith in humanity. Honestly, I think if you ran a poll, most people would think that the Hawkeye graphic represents what actually happened, when in reality it's an operators interpretation of what most probably happened.

Except the next time the above example I quoted happens on championship point.

And also just as we (Well, BWF, after much cajoling) implemented Hawkeye after far, far too many inaccurate line calls, if not constantly improved, so too will Hawkeye share the same fate. If you believe that standards are there to evolve and improve, you'd have to agree that at some stake, Hawkeye will also be deemed unacceptable.

This was not the first time, and it will not be the last either. Recall the last rally of MSF at All England, Chong Wei - Tago - in 2010, there was no IRS, however, there was clear green space seen with a naked eye, even in this low res video - .

Be all this, it helps to give the BWF and the competition authority some benefit of doubt, cut them some slack. When ITF started using Hawk-Eye in 2005, it was not until 2008 that the technology was consolidated. In this period, some tournaments has 2 challenges per match and some unlimited. At least in badminton, we now have grown-up to 2 unsuccessful challenges per game per team, barring some situations when the umpire does not accept the challenge for review. There is a circular somewhere why it is 2 challenges. And the communication from umpire (vocabulary) has become understandable, none of the thumbs down infantile-silliness. When a problem is identified, then a solution will be forthcoming; to define the problem takes a few iterations.

 

[Charlie-SWUK]

I think it's fine in your example.



Trace the trajectory of the shuttle, the back service line, and then make them intersect.

 

[stan2000]

My main issue is that the entire system is being touted as an infallible system. People have it in their minds that it's incredibly accurate and faultless. Remember those line calls that are called 'in', and the graphic zooms in - sometimes multiple times - showing the shuttle clipping the lines by millimeters? This is complete rubbish. What they're showing is a lie.

There's also two other points.

Firstly, That the cork on the end of the shuttle is a dome, so it's level of deformation is entirely speed and angle dependant. But what do we get via Hawkeye? A perfectly round dot of the same size, every time. Regardless of how the shuttle hits the ground.

Secondly, and I know this has been discussed before, but I have a hunch that it might actually be possible for the shuttles feathers to hit the ground at the same time if not milliseconds earlier than the cork. I haven't got a shuttle handy, but I suspect it just might be possible if the angle it's struck at is the same as that of the angle of the feathers. If so, whats the rules on that? At the moment Hawkeye places a round dot indicating whether it's in or not, but that is patently obvious not what happens.

Mathematixally usibg a v squares air resistance model and the dimensions and angle of the feather it is not possible fot the featgers to touch the ground befire the cork

 

[juneau-AK]

Without changing the subject too much, it is very possible for the feathers to touch court before cork, especially at lower velocities.This is more of physics than maths. Any part of the shuttle touching of the court determines the position where it 'landed' which means that the cork does not have to touch for the shuttle to be considered IN.

The reason is same as when players do the chop serve, or a net tumbler - it takes time. The shuttle axis undergoes a damped oscillation until it align along the velocity vector. If you visit any high performance baddy club, this is what the top MD players practice - they try to increase the oscillating time. There are several variations that you would see, just look for the the Korean, the Danish and the Indonesian pairs.

There is a fascinating read that was published in a scientific journal, if anyone is interested in looking at what happens immediately after the Big Bang, I mean, the big hit. Have a look-see here - http://bit.ly/2WUmlAw.

 

[Ouchie]

Good timing to resurface this (sensationalist) thread...see how Hawk-eye works.


"Really accurate models"
"See exactly where the shuttle landed"
"Whoever deserves to win the match, wins the match"

 

[Tennyson]

Since human decisions are always controversial, it is best to leave it to a machine (designed by infallible humans) to be the neutral, unselfish and fair judge

 

[event]

Good timing to resurface this (sensationalist) thread...see how Hawk-eye works.


"Really accurate models"
"See exactly where the shuttle landed"
"Whoever deserves to win the match, wins the match"

First of all, I don't think it's fair to call the thread sensationalist. The OP is clear about simply calling for scrutiny and never advocated for throwing out Hawk-Eye. Second, this video does not show how the system works. Rather, it shows a little of how the humans who operate it go about their jobs before and during the week of a tournament. juneau-AK's response above has more detail on how it works than the video, which never mentions anything about how the software works. The original post lends credence to the rumours and suspicion that have surrounded Hawk-Eye technology for years. Namely, that someone in a booth looks at a replay to determine whether it's in or out, then uses a mouse to select a spot on the court and Hawk-Eye technology generates a graphic for the benefit of the television viewers and those looking at the big screen. If juneau-AK is correct and it is predictive software, then it is arguably more worrying because, as the OP points out, if Hawkeye thinks something that a replay suggests is one cm out is really several centimetres out, then it could read something in as out.

It doesn't mean the video isn't interesting but I don't think it sheds much light on the 'it didn't look anywhere near that far out on the replay' issue.