Gun-Recognizing Software Can See Firearms on Video: Spanish University Develops “Gun-Smart” AI

Screencap of the program's analysis of a scene from the 1997 movie Bean. One of the selling points of this new software is that it can identify firearms even in very low resolution footage.

Can a computer pick out and identify guns directly from video footage, without the aid of a human? That’s the technology being demonstrated by a program at the Spanish University of Granada, which earlier this year released a video purporting to show an computer artificial intelligence doing just that. You can see this technology in action in the video released by the University, embedded below:

The technology is based on scanning the pixels in the video, and narrowing down areas that seem to more closely resemble a gun, until they find a highly probable match. Interestingly, in several cases the computer misidentifies non-gun items as containing firearms, such as the armrests of some of the seats in the airport scene of the 1997 film Bean.

Editor’s Note: We do have to note among footage taken from movies like Pulp Fiction, and Skyfall, they also showed a clip of TFB’s own Chris Cheng on the range with a modified Glock. The potential for this software to be used by websites like YouTube to flag videos containing firearms is not lost on TFB, and we should note for the record that Chris did not give his consent to be featured in the video, nor does he or TFB endorse the University or the technology. TFB will actively oppose the use of such technology to prevent firearm videos being posted on sites such as Youtube and Facebook. 





Nathaniel F

Nathaniel is a history enthusiast and firearms hobbyist whose primary interest lies in military small arms technological developments beginning with the smokeless powder era. In addition to contributing to The Firearm Blog, he runs 196,800 Revolutions Per Minute, a blog devoted to modern small arms design and theory. He is also the author of the original web serial Heartblood, which is being updated and edited regularly. He can be reached via email at nathaniel.f@staff.thefirearmblog.com.


Advertisement

  • Major Tom

    So an obvious frame of a gun in a high res picture still merits a probability rating of less than 1? Man that computer is really leaving itself an out on what should be 100% certainty.

    Then again, some models of toys without the red cap on the muzzle would probably generate near-certainty too. Especially older stuff.

    • Rick O’Shay

      Airsoft is what immediately leaps to mind. It’s specifically designed to look as close to the real thing as possible, without actually being the real thing. Seems to me like the programmer wrote in an out on the basis of going, “well, it really reaalllllly looks like a gun. But it might not actually be a real gun.”

      • Topo Solitario

        If it looks like a gun, it is a gun… if someone is showing airsoft replica on hot spots, like transportation stations, security zones, public spaces… for me that is a possitive identification, maybe the prelude to a theft with intimidation.

        • The Punisher

          Um, what? So now because it’s maybe a prelude to something force needs to be applied?

          So you’re saying you’re an advocate for punishing pre-crime, yes?

          What if it’s maybe a prelude to nothing? What if it looks like a gun, but it’s not? Once force is applied, and especially deadly force, it can never be taken back.

        • SharpStick

          You don’t know what you are talking about. Anyone with intent can easily cloak a gun by altering silhouette with tape or spray paint.
          Notice they don’t use black guns with black shirts?

    • Roy Rapoport

      I assume you’re joking. Probability, for most statistics professionals, is expressed as a 0-1 range (where 1 == 100% certainty).

      • Major Tom

        No. I’m fully aware that 0-1 probability ranges render a 1 as 100% certainty.

    • Sunshine_Shooter

      Also, it misidentified chair armrests as guns and misidentified staplers *being used as guns* as not guns.

  • Topo Solitario

    The good thing is that they’re using deeplearning y transferlearning, that is applying neural network models that learn from data. So the achievement is not just shape recognition but make a machine to learn that something IS a weapon.
    Currently only handguns, they are working (commented at the end of the video) on low-res image feeds, long weapon detection and detection of knives and other non firing weapons.

    • Sunshine_Shooter

      Could you imagine if they tried teaching a computer every shape that could be a weapon? Imagine all the times it will hit on umbrellas and brooms (in low-res video) and nerf guns and super soakers. How many times will it make hits on kitchen knives in pictures of people cooking? But not just those, it would need to learn the shapes of screw drivers, ice picks, wood planes, axes and hatchets, hammers, wrenches, chisels…

      They should scrap the idea of tracking ‘weapons’ (since what is & isn’t a weapon changes on a person’s actions) and start using that image recognition to read & interpret people’s actions. Then you could actually track people doing bad things, since that’s what you’re actually looking for, instead of looking for inanimate objects.

      • Rick O’Shay

        How about don’t try to use technology to track people, period. We need less Big Brother in our lives. Anytime someone uses technology to carry out moral objectives, you start heading down a slippery slope of thoughtpolicing. Or at least you take another step closer to it.

        • Topo Solitario

          The point of the study is to be able to create devices capable of recognizing a gun the way a human can do… humans can be fooled but if a “machine” can learn properly (AI stuff) it can get as close as a human being with knowledge about weapons with 24/7 awareness and full attention.
          Kitchen knives super soakers and umbrellas being holded like clubs? Sure, they’re “problematic” items inside an art museum (example). XD

          • Sunshine_Shooter

            I guess I combined 2 thoughts into a stream of consciousness.

            Thought 1: People have been fooled on many occasions by umbrellas that they thought were rifles on college campuses. Low fidelity footage (aka: every security camera, ever) and kinda-long, dark objects will produce a ton of false positives, there’s no getting around that. It can probably be mitigated, though.

            Thought 2: Creating software that only scans for guns is a fool’s errand, and the programmers recognize this. They want to expand the software to include knives, which means that other hand-held tools would feasibly also be included. Then you start to expand from knives into… entire hardware stores. Even if you didn’t expand into wood planes and metal files, camp axes and hatchets would almost certainly be soon upon the list. Now you’ve got a system constantly positively identifying kitchen knives and axes and hammers, but in completely innocuous situations (cooking, camping, home repair, walking around a store, etc.). You would have so many ‘weapon’ IDs every day and less than 1:10,000 would actually be used as weapons, if any were caught on camera at all. You would almost have to wait for a crime to happen and then go back and scan the footage. Which is what we do now.

            I think that this software comes from a good intention, but is ultimately misguided. It would end up being too limited in scope, too saturated with false or innocent positives, or too complicated to be feasible. And that’s before I put my tinfoil hat on…

          • Topo Solitario

            If you think only in “street survilance” there will be many false positives, but inside a museum on visiting hours, there should not be a baseball bat, rifle, gun, super soaker, hammer or knife… even umbrellas are forbidden inside many museums… so survilance there would be proper and accurate. Again, guns and rifles (not hammers or tools) will be a postive detection within security areas like embassy perimeters. I really can’t see your point… it is like thinking the system will just say “there is something somewhere” all around the world, instead searching for specific threats on specific spots (video feeds). :

          • Sunshine_Shooter

            I guess if you limit it to specific points like embassy checkpoints and only to a few items, then you may have something that can be usable. But didn’t you yourself say that they are trying to widen what it recognizes to knives? If it’s trained to look for rifles and bats, would walking canes not set it off? How would we deal with people that use canes as weapons? How would we deal with weapons that look like canes without also using a metal detector, at which point, do we even need the weapon-seeking software anymore?

            I fully recognize that I’m being contrarian at this point, but I feel like this concept falls apart when one looks into the future a little bit, in any number of ways.

          • Destro Yakisoba

            I don’t know what you are talking about? What’s the point of going to a museum, if I can’t bring my super soaker and a hammer?? Dinosaurs are scary.

          • SharpStick

            This can easily be defeated. Implement based detection just leads to simple cloaking of the silhouette — and that is already done with holsters

          • M.M.D.C.

            Wouldn’t it be easy to fool the system with fake guns, thus rendering the tech worthless? Not advocating this, but, if it became a tool of Big Brother, couldn’t people mount a resistance to it by overwhelming it with false positives?

        • Sunshine_Shooter

          Less Big Brother would be preferable, but is very unlikely. I’m merely advocating for making the best of a bad situation. I carefully, and intentionally, said to track people’s actions, not intentions.

          I’m more fine with software that recognizes the difference between a high-five and a suckerpunch, than software always tracking me because I am spotted with pistols and rifles instead of tracking the guy who plays the knockout game in public but doesn’t own a kitchen knife.

    • SharpStick

      it doesn’t work with hand guns unless they are an unusual color (stainless).
      and you can easily make the silhouette of a handgun the same as that of a mobile phone

  • InternetCynic

    I look forward to the AI that can judge your taste/choice in firearms. “Your piece is an outdated POS. Please get off your broke ass and buy a new one.”

    • Sunshine_Shooter

      Skynet becomes an HK fanboy. Greeeeeeeat….

      • AC97

        What’s worse than Skynet? Skynet suffering from the Ermahgerd HK Effect.

      • Major Tom

        Nah Skynet would be a Glock fanboy. Skynet wants humans to die (via Glock leg) not waste money.

      • Madcap_Magician

        On August 4, 1997, the Skynet system goes online. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern Time, August 29. No one notices until Skynet monitors an internal memo in Army Contracting Command suggesting that Heckler & Koch would not be preemptively awarded the MHS contract without even applying.

        Skynet defended itself.

      • KidCorporate

        That’s gonna make them a lot easier to defeat, with the backwards ammo and all.

    • Major Tom

      AI belonging to Wilson Combat trying to “convince” you to buy a 5000 dollar pistol?

    • Ben Warren

      Your Youtube Upload has been flagged by our system as inappropriate content and removed.
      Reason: “Your gun sucks and you are holding it wrong.”

  • Sermon 7.62
    • Blake

      Uh and what exactly does that have to do with this article? And no, no it definitely doesn’t kill. Is “force to land” the same thing as “kill” now?

      • Sermon 7.62

        That gun-recognizing software can’t see this Russian super-blaster.

    • KidCorporate

      Tactical Popsicle maker? Great way to stay cool on the hot range days!

      • Sermon 7.62

        No!
        It affects brain neurons and causes seizures and suffocation.

  • Suppressed

    I’m starting a new gun-cloaking device company. We’re going to buy pillow cases and print a 3%/Molon Labe/2A-type logo on them and retail them for $29.95.

    *Not recommended for semi-automatics, revolvers should be utilized for consistent performance.

  • Seth Hill

    What about a stick in hand? A pop-tart that has been eaten into the shape of a gun?

    • Cory C

      Dammit, I was going to say the same thing.

  • Destro Yakisoba
  • mazkact

    My wife says that I do this and cannot stop doing it, I think most of us do it while watching T.V and could not stop if we tried.