The Infona portal uses cookies, i.e. strings of text saved by a browser on the user's device. The portal can access those files and use them to remember the user's data, such as their chosen settings (screen view, interface language, etc.), or their login data. By using the Infona portal the user accepts automatic saving and using this information for portal operation purposes. More information on the subject can be found in the Privacy Policy and Terms of Service. By closing this window the user confirms that they have read the information on cookie usage, and they accept the privacy policy and the way cookies are used by the portal. You can change the cookie settings in your browser.
In this work, we apply the idea of conditional computation to the gated recurrent unit (GRU), a type of recurrent activation function. With slight modifications to the GRU, the number of floating point operations required to calculate the feed-forward pass through the network may be significantly reduced. This allows for more rapid computation, enabling a trade-off between model accuracy and model...
Catastrophic forgetting is a problem encountered with neural networks as well as other learning systems whereby past representations are lost as new representations are learned. It has been shown that catastrophic forgetting can be mitigated in neural networks by using a neuron selection technique, dubbed "cluster-select," which performs online clustering over the network inputs to partition...
Deep Machine Learning (DML) algorithms have proven to be highly successful at challenging, high-dimensional learning problems, but their widespread deployment is limited by their heavy computational requirements and the associated power consumption. Analog computational circuits offer the potential for large improvements in power efficiency, but noise, mismatch, and other effects cause deviations...
Catastrophic forgetting is a fundamental problem with artificial neural networks (ANNs) in which learned representations are lost as new representations are acquired. This significantly limits the usefulness of ANNs in dynamic or non-stationary settings, as well as when applied to very large datasets. In this paper, we examine a novel neural network architecture which utilizes online clustering for...
Catastrophic forgetting is a well studied problem in artificial neural networks in which past representations are rapidly lost as new representations are constructed. We hypothesize that such forgetting occurs due to overlap in the hidden layers, as well as the global nature in which neurons encode information. We introduce a novel technique to mitigate forgetting which effectively minimizes activation...
Direct processing of raw high-dimensional data such as images and video by machine learning systems is impractical both due to prohibitive power consumption and the “curse of dimensionality,” which makes learning tasks exponentially more difficult as dimension increases. Deep machine learning (DML) mimics the hierarchical presentation of information in the human brain to achieve robust automated feature...
An analog clustering circuit is presented. It is capable of inferring the underlying pattern and extracting the statistical parameters from the input vectors, as well as providing measures of similarity based on both mean and variance. A floating-gate analog memory provides non-volatile storage. A current-mode distance computation, a time-domain loser-take-all and a memory adaptation circuit implement...
Catastrophic forgetting (or catastrophic interference) in supervised learning systems is the drastic loss of previously stored information caused by the learning of new information. While substantial work has been published on addressing catastrophic forgetting in memoryless supervised learning systems (e.g. feedforward neural networks), the problem has received limited attention in the context of...
In many unsupervised learning applications both spatial and temporal regularities in the data need to be represented. Traditional clustering algorithms, which are commonly employed by unsupervised learning engines, lack the ability to naturally capture temporal dependencies. In supervised learning methods, temporal features are often learned through the use of a feedback (or recurrent) signal. Drawing...
This paper introduces a sequentially motivated approach to processing streams of images from datasets with low memory demands. We utilize fuzzy clustering as an incremental dictionary learning scheme and explain how the corresponding membership functions can be subsequently used in encoding features for image patches. We focus on replicating the codebook learning and classification stages from an...
Ensembles of neural networks have been the focus of extensive studies over the past two decades. Effectively encouraging diversity remains a key element in yielding improved performance from such ensembles. Negatively correlated learning (NCL) has emerged as a promising framework for concurrently training an ensemble of learners while emphasizing the cooperation among them. The NCL methodology relies...
Graphs play a role in many semi-supervised learning algorithms, where unlabeled samples are used to find useful structural properties in the data. Dimensionality reduction and regularization based on preserving smoothness over a graph are common in these settings, and they perform particularly well if proximity in the original feature space closely reflects similarity in the classification problem...
In this paper we present the fixed expansion layer (FEL) feedforward neural network designed for balancing plasticity and stability in the presence of non-stationary inputs. Catastrophic interference (or catastrophic forgetting) refers to the drastic loss of previously learned information when a neural network is trained on new or different information. The goal of the FEL network is to reduce the...
Visual attention is the cognitive process of directing our gaze on one aspect of the visual field while ignoring others. The mainstream approach to modeling focal visual attention involves identifying saliencies in the image and applying a search process to the salient regions. However, such inference schemes commonly fail to accurately capture perceptual attractors, require massive computational...
Deep machine learning is an emerging framework for dealing with complex high-dimensionality data in a hierarchical fashion which draws some inspiration from biological sources. Despite the notable progress made in the field, there remains a need for an architecture that can represent temporal information with the same ease that spatial information is discovered. In this work, we present new results...
This paper details a methodology and preliminary results for applying a hierarchy of clustering units to mammographic image data. The identification of patients with breast cancer through the detection of microcalcifications and masses is a demanding classification problem; minimal false negatives are desired while simultaneously avoiding false positives that cause unnecessary cost to patients and...
This paper details a methodology and preliminary results for applying a hierarchy of clustering units to mammographic image data. The identification of patients with breast cancer through the detection of microcalcifications and masses is a demanding classification problem; minimal false negatives are desired while simultaneously avoiding false positives that cause unnecessary cost to patients and...
Clustering is a pivotal building block in many data mining applications and in machine learning in general. Most clustering algorithms in the literature pertain to off-line (or batch) processing, in which the clustering process repeatedly sweeps through a set of data samples in an attempt to capture its underlying structure in a compact and efficient way. However, many recent applications require...
Set the date range to filter the displayed results. You can set a starting date, ending date or both. You can enter the dates manually or choose them from the calendar.