The Infona portal uses cookies, i.e. strings of text saved by a browser on the user's device. The portal can access those files and use them to remember the user's data, such as their chosen settings (screen view, interface language, etc.), or their login data. By using the Infona portal the user accepts automatic saving and using this information for portal operation purposes. More information on the subject can be found in the Privacy Policy and Terms of Service. By closing this window the user confirms that they have read the information on cookie usage, and they accept the privacy policy and the way cookies are used by the portal. You can change the cookie settings in your browser.
In this paper, the filtering problem for a large class of continuous-time, continuous-state stochastic dynamical systems is considered. Inspired by recent advances in asymptotically-optimal sampling-based motion planning algorithms, such as the PRM* and the RRT*, an incremental sampling-based algorithm is proposed. Using incremental sampling, this approach constructs a sequence of Markov chain approximations,...
We consider a service provider that accommodates two classes of users: primary users (PUs) and secondary users (SUs). SU demand is elastic to price whereas PU demand is inelastic. When a PU arrives to the system and finds all channels busy, it preempts an SU unless there are no SUs in the system. Call durations are exponentially distributed and their means are identical. We study the optimal pricing...
Traditional Markov reliability model used to predict the total number of errors in the software and the failure interval, but did not consider the impact of the severity of the error. However, in some cases, such as software trustworthiness, we must consider not only the total number of software errors and failure interval, but also consider the severity of the error on the impact of the credibility...
We show how a Markov chain provides a simple representation of the underlying character of a Blue versus Red battle engagement. Fixed time-step simulation provides a natural practical implementation of such a representation. We demonstrate how such an implementation can be optimally tuned to model and capture the most important aspects of a given battle whilst still enabling simulations to be carried...
We are interested in risk constraints for discrete time Markov decision processes (MDPs). Starting with the average reward case, we argue that stochastic dominance constraints are natural risk constraints for MDPs. Specifically, we constrain the empirical distribution of reward to dominate a benchmark distribution in the increasing concave stochastic order. We argue that the optimal policy for the...
A probabilistic model for the diffusion of heat on one-dimensional spaces is developed. Specifically, grid points are arbitrarily placed on the real line and the heat particles are assumed to jump between these grid points in continuous time. This random walk by the heat particles is represented by a continuous-time Markov chain and state-transition intensities depends on the underlying, possibly...
The paper examines existence of equilibrium price distributions in energy markets with real-time pricing and consumers with time-flexible demands. Previous works have examined consumer optimal policies for shifting time-flexible loads up to a deadline, in response to an exogenous and stochastic price process. It is shown here that under some mild assumptions on the stochastic price process and the...
In this paper, we consider the problem of designing a feedback policy for a discrete time stochastic hybrid system that should be kept operating within some compact set A. To this purpose, we introduce an infinite-horizon discounted average reward function, where a negative reward is associated to the transitions driving the system outside A and a positive reward to those leading it back to A. The...
In this paper, we address the problem of optimal control in a service-oriented manufacturing (SOM) system. The optimal dynamic admission control of the product orders and the optimal dynamic production control of products are examined simultaneously. We formulate the integrated optimal control problem as a continuous Markov decision process. and characterize the integrated optimal dynamic control...
Sampling from large graphs is an area of great interest, especially since the emergence of huge structures such as Online Social Networks (OSNs) and the World Wide Web (WWW). These networks, when viewed as graphs, often contain hundreds of millions of vertices and billions of edges. The large scale properties of a network can be summarized in terms of parameters of the underlying graph, such as the...
Discrete Time Markov Chains (DTMCs) and Continuous Time Markov Chains (CTMCs) are often used to model various types of phenomena, such as, for example, the behavior of software products. In that case, Markov chains are widely used to describe possible time-varying behavior of “self-adaptive” software systems, where the transition from one state to another represents alternative choices at the software...
This paper studies multi-objective reinforcement learning problems in which an agent gains multiple rewards. In ordinary multi-objective reinforcement learning methods, only a single Pareto optimal policy is acquired by the scalarizing method which uses the weighted sum of the reward vector, and therefore different Pareto optimal policies are acquired by changing the weight vector and by performing...
A secret key agreement setup between three users is considered in which each of the users 1 and 2 intends to share a secret key with user 3 and users 1 and 2 are eavesdroppers with respect to each other. The three users observe i.i.d. outputs of correlated sources and there is a generalized discrete memoryless multiple access channel (GDMMAC) from users 1 and 2 to user 3 for communication between...
This paper presents a computationally efficient method to forecast floods stochastically. The main purpose of the method is to encapsulate prior knowledge as off-line calculations leading to earliest possible warnings. The computational efficiency is improved through exploiting the stereotypical features of hydrology and its dependence on topography by combining the parallel water-flow processes into...
Efforts are underway in improving power conservation in WLAN devices and thus increasing the duration between recharging the battery. In order to evaluate the performance of new energy saving methods, there is a need for analytical tools. This paper proposes a simple analytical model that provides the average power consumption of a WLAN device. The proposed model is based on Markov models that provide...
A wide range of real life systems are modeled by queueing systems with finite capacity buffers. There are well established numerical procedures for the analysis of these queueing models when the load islower or higher than the system capacity, but these numerical methods become unstable as the loadgets close to the system capacity. We present simple modifications of the standard computational methodswhich...
Consider a Markov modulated fluid queue with multiple layers separated by a finite number of boundaries, where each layer is characterized by its own set of matrices. In the past, matrix analytic methods have been devised to determine the stationary behavior of such a fluid queue for no-resistance, sticky and repellent boundaries. In this paper we extend this approach by allowing general phase transitions...
The paper presents a research on transitive probabilities of a binary Markov logic. Analytical dependencies are obtained of such transitions. A Markov interpretation is introduced of propositional-logic formulas using only two binary Markov matrices, two rules and a suitably selected negation of binary vectors. Formulas are obtained for final transitive probabilities for each of the two selected binary...
One of the issues to be resolved in social recommender systems is the identification of opinion leaders in a network. Finding effective people in societies has been a key question for many groups, e.g., marketers. The research undertaken in this paper focuses on finding important nodes in a network based on their behaviour as well as the structure of the network. This paper views the propagation of...
Though, Wireless Sensor Networks have many successful applications in military, smart building, habitat monitoring, etc. it just begun to show its superiority in the medical domain in recent years. Deployment of WSNs in medical domain exert many more stringent requirements on the capability of sensing and in-networking processing, due to the nature of high accessibility and the privacy sensitive requirement...
Set the date range to filter the displayed results. You can set a starting date, ending date or both. You can enter the dates manually or choose them from the calendar.