Connect with us


Google’s DeepMind asks what it means for AI to fail



There’s been years of study placed in the problem of how to make artificial intelligence “robust” to attack and less prone to failure. Yet the field is still coming to grips with what failure in AI actually means, as pointed out by a blog post this week from the DeepMind unit of Google.

The missing element may seem obvious to some: it would really help if there was more human involvement in setting the boundary conditions for how neural networks are supposed to function.

Researchers Pushmeet Kohli, Sven Gowal, Krishnamurthy, Dvijotham, and Jonathan Uesato have been studying the problem, and they identify much work that remains to be done, which they sum up under the title “Towards Robust and Verified AI: Specification Testing, Robust Training, and Formal Verification.”

There’s a rich history of verification testing for computer programs, but those approaches are “not not suited for modern deep learning systems.” 

Also: MIT ups the ante in getting one AI to teach another 

Why? In large part because scientists are still learning about what it means for a neural network to follow the “specification” that was laid out for it. It’s not always clear what the specification even is.

“Specifications that capture ‘correct’ behavior in AI systems are often difficult to precisely state,” the authors write. 

Google’s DeepMind proposes ways to set a bound on the kinds of outputs a neural network can produce, to keep it from doing the wrong thing. 


The notion of a “specification” comes out of the software world, the DeepMind researchers observe. It is the intended functionality of a computer system. 

As the authors wrote in a post in December, in AI, there may not be just one spec, there may be at least three. There is the “ideal” specification, what the system’s creators imagine it could do. Then there is the “design” specification, the “objective function” explicitly optimized for a neural network. And, lastly, there is the “revealed” specification, the way that the thing actually performs. They call these three specs, which all can vary quite a bit from one another, the wish, the design, and the behavior. 

Designing artificial neural networks can be seen as how to close the gap between wish, design and behavior. As they wrote in the December essay, “A specification problem arises when there is a mismatch between the ideal specification and the revealed specification, that is, when the AI system doesn’t do what we’d like it to do. ”

Also: Google ponders the shortcomings of machine learning

They propose various routes to test and train neural networks that are more robust to errors, and presumably more faithful to specs. 

One approach is to use AI itself to figure out what befuddles AI. That means using a reinforcement learning system, like Google’s AlphaGo, to find the worst possible ways that another reinforcement learning system can fail? 

The authors did just that, in a paper published in December. “We learn an adversarial value function which predicts from experience which situations are most likely to cause failures for the agent.” The agent in this case refers to a reinforcement learning agent. 

“We then use this learned function for optimisation to focus the evaluation on the most problematic inputs.” They claim that the method leads to “large improvements over random testing” of reinforcement learning systems.

Must read

Another approach is to train a neural network to avoid a whole range of outputs, to keep it from going entirely off the rails and making really bad predictions. The authors claim that a “simple bounding technique,” something called “interval bound propagation,” is capable of training a “verifiably robust” neural network. That work won them a “best paper” award at the NeurIPS conference last year.

They’re now moving beyond just testing and training a neural network to avoid disaster, they’re also starting to find a theoretical basis for a guarantee of robustness. They approached it as an “optimisation problem that tries to find the largest violation of the property being verified.” 

Despite those achievements, at the end of the day, “much work is needed,” the authors write “to build automated tools for ensuring that AI systems in the real world will do the ‘right thing’.” 

Some of that work is to design algorithms that can test and train neural networks more intensely. But some of it probably involves a human element. It’s about setting the goals — the objective function — for AI that matches what humans want. 

“Building systems that can use partial human specifications and learn further specifications from evaluative feedback would be required,” they write, “as we build increasingly intelligent agents capable of exhibiting complex behaviors and  acting in unstructured environments.”

Previous and related coverage:

What is AI? Everything you need to know

An executive guide to artificial intelligence, from machine learning and general AI to neural networks.

What is deep learning? Everything you need to know

The lowdown on deep learning: from how it relates to the wider field of machine learning through to how to get started with it.

What is machine learning? Everything you need to know

This guide explains what machine learning is, how it is related to artificial intelligence, how it works and why it matters.

What is cloud computing? Everything you need to know about

An introduction to cloud computing right from the basics up to IaaS and PaaS, hybrid, public, and private cloud.

Related stories:

Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *


Work from Home Security



Spin Master is a leading global children’s entertainment company that invents toys and games, produces dozens of television and studio series that are distributed in 160 countries, and creates a variety of digital games played by more than 30 million children. What was once a small private company founded by childhood friends is now a public global supply chain with over 1,500 employees and 28 offices around the world.

Like most organizations in 2020, Spin Master had to adapt quickly to the new normal of remote work, shifting most of its production from cubicles in regional and head offices to hundreds of employees working from home and other remote locations.

This dramatic shift created potential security risks, as most employees were no longer behind the firewall on the corporate network. Without the implementation of hardened endpoint security, the door would be open for bad actors to infiltrate the organization, acquire intellectual property, and ransom customer information. Additionally, the potential downtime caused by a security breach could harm the global supply chain. With that in mind, Spin Master created a self-imposed 30-day deadline to extend its network protection capabilities to the edge.

Key Findings:

  • Think Long Term: The initial goal of establishing a stop-gap work-from-home (WFH) and work-from-anywhere (WFA) strategy has since morphed into a permanent strategy, requiring long-term solutions.
  • Gather Skills: The real urgency posed by the global pandemic made forging partnerships with providers that could fill all the required skill sets a top priority.
  • Build Momentum: The compressed timeline left no room for delay or error. The Board of Directors threw its support behind the implementation team and gave it broad budget authority to ensure rapid action, while providing active guidance to align strategy with action.
  • Deliver Value: The team established two key requirements that the selected partner must deliver: implementation support and establishing an ongoing managed security operations center (SOC).
Continue Reading


Key Criteria for Evaluating Privileged Access Management



Privileged Access Management (PAM) enables administrative access to critical IT systems while minimizing the chances of security compromises through monitoring, policy enforcement, and credential management.

A key operating principle of all PAM systems is the separation of user credentials for individual staff members from the system administration credentials they are permitted to use. PAM solutions store and manage all of the privileged credentials, providing system access without requiring users to remember, or even know, the privileged password. Of course, all staff have their own unique user ID and password that they use to complete everyday tasks such as accessing email and writing documents. Users who are permitted to handle system administration tasks that require privileged credentials log into the PAM solution, which provides and controls such access according to predefined security policies. These policies control who is allowed to use which privileged credentials when, where, and for what tasks. An organization’s policy may also require logging and recording of the actions undertaken with the privileged credentials.

Once implemented, PAM will improve your security posture in several ways. The first is by segregating day-to-day duties from duties that require elevated access, reducing the risk of accidental privileged actions. Secondly, automated password management reduces the possibility that credentials will be shared while also lowering the risk if credentials are accidentally exposed. Finally, extensive logging and activity recording in PAM solutions aids audits of critical system access for both preventative and forensic security.

How to Read this Report

This GigaOm report is one of a series of documents that helps IT organizations assess competing solutions in the context of well-defined features and criteria. For a fuller understanding consider reviewing the following reports:

Key Criteria report: A detailed market sector analysis that assesses the impact that key product features and criteria have on top-line solution characteristics—such as scalability, performance, and TCO—that drive purchase decisions.

GigaOm Radar report: A forward-looking analysis that plots the relative value and progression of vendor solutions along multiple axes based on strategy and execution. The Radar report includes a breakdown of each vendor’s offering in the sector.

Vendor Profile: An in-depth vendor analysis that builds on the framework developed in the Key Criteria and Radar reports to assess a company’s engagement within a technology sector. This analysis includes forward-looking guidance around both strategy and product.

Continue Reading


Adventist Risk Management Data Protection Infrastructure



Companies always want to enhance their ability to quickly address pressing business needs. Toward that end, they look for new ways to make their IT infrastructures more efficient—and more cost effective. Today, those pressing needs often center around data protection and regulatory compliance, which was certainly the case for Adventist Risk Management. What they wanted was an end-to-end, best-in-class solution to meet their needs. After trying several others, they found the perfect combination with HYCU and Nutanix, which provided:

  • Ease of deployment
  • Outstanding ROI
  • Overall TCO improvement

Nutanix Cloud Platform provides a software-defined hyperconverged infrastructure, while HYCU offers purpose-built backup and recovery for Nutanix. Compared to the previous traditional infrastructure and data protection solutions in use at Adventist Risk Management, Nutanix and HYCU simplified processes, speeding day-to-day operations up to 75%. Now, migration and update activities typically scheduled for weekends can be performed during working hours and help to increase IT staff and management quality of life. HYCU further increased savings by providing faster and more frequent points of recovery as well as better DR Recovery Point Objective (RPO) and Recovery Time Objective (RTO) by increasing the ability to do daily backups from one to four per day.

Furthermore, the recent adoption of Nutanix Objects, which provides secure and performant S3 storage capabilities, enhanced the infrastructure by:

    • Improving overall performance for backups
    • Adding security against potential ransomware attacks
    • Replacing components difficult to manage and support

In the end, Nutanix and HYCU enabled their customer to save money, improve the existing environment, and, above all, meet regulatory compliance requirements without any struggle.

Continue Reading