Adaptive Representations for Reinforcement Learning by Shimon Whiteson PDF

By Shimon Whiteson

ISBN-10: 3642139310

ISBN-13: 9783642139314

This publication offers new algorithms for reinforcement studying, a kind of desktop studying within which an self sufficient agent seeks a keep an eye on coverage for a sequential choice activity. when you consider that present equipment mostly depend on manually designed resolution representations, brokers that immediately adapt their very own representations have the capability to dramatically enhance functionality. This ebook introduces novel methods for instantly gaining knowledge of high-performing representations. the 1st strategy synthesizes temporal distinction equipment, the conventional method of reinforcement studying, with evolutionary tools, which could research representations for a wide category of optimization difficulties. This synthesis is complete through customizing evolutionary easy methods to the online nature of reinforcement studying and utilizing them to adapt representations for price functionality approximators. the second one procedure instantly learns representations in accordance with piecewise-constant approximations of worth features. It starts off with coarse representations and steadily refines them in the course of studying, examining the present coverage and price functionality to infer the simplest refinements. This e-book additionally introduces a singular procedure for devising enter representations. this technique addresses the function choice challenge by means of extending an set of rules that evolves the topology and weights of neural networks such that it evolves their inputs too. as well as introducing those new tools, this publication provides huge empirical leads to a number of domain names demonstrating that those suggestions can considerably increase functionality over tools with guide representations.

Show description

Read Online or Download Adaptive Representations for Reinforcement Learning PDF

Best nonfiction_6 books

New PDF release: The Turkish Army of 1812 from the book by Thomas McLean

This infrequent set of prints is nearly mythical between students of the army uniform. those prints have been made of genuine watercolor work of the topics in Turkey via an itinerant French painter named de Molleville. One wonders if he have been possibly a secret agent despatched via Napoleon to evaluate the army may possibly of Turkey?

Additional info for Adaptive Representations for Reinforcement Learning

Example text

0. 6. 36 4 Evolutionary Function Approximation Uniform Moving Average Score Per Episode Uniform Moving Average Score Per Episode 0 -10000 -50 -10500 NEAT+Q -100 -11000 -150 -12000 Score -200 Score NEAT+Q -11500 NEAT -250 -300 -12500 NEAT -13000 Q−Learning -350 -13500 -400 -14000 Q−Learning -450 -14500 -500 -15000 0 200 400 600 Episode (x1000) (a) Mountain Car 800 1000 0 200 400 600 800 1000 Episode (x1000) (b) Server Job Scheduling Fig. 1 A comparison of the performance of NEAT, NEAT+Q, and Q-learning with the best of 24 different manually designed neural network function approximators in the mountain car and server job scheduling domains.

42 4 Evolutionary Function Approximation Fig. 4 Comparing Darwinian and Lamarckian Approaches As described in the beginning of this chapter, evolutionary function approximation can be implemented in either a Darwinian or Lamarckian fashion. The results presented so far all use the Darwinian implementation of NEAT+Q. However, it is not clear that this approach is superior even though it more closely matches biological systems. In this section, we compare the two approaches empirically in both the mountain car and server job scheduling domains.

McGovern et al. (91) use reinforcement learning for CPU instruction scheduling but aim only to minimize completion time. One method that can be adapted to the server job scheduling task is the generalized cμ rule (95), in which the server always processes at time t the oldest job of that type k which maximizes Ck (ok )/pk , where Ck is the derivative of the cost function for job type k, ok is the age of the oldest job of type k and pk is the average processing time for jobs of type k. Since in our simulation all jobs require unit time to process and the cost function is just the additive inverse of the utility function, this algorithm is equivalent to processing the oldest job of that type k that maximizes −Uk (ok ), where Uk is the derivative of the utility function for job type k.

Download PDF sample

Adaptive Representations for Reinforcement Learning by Shimon Whiteson

by Joseph

Rated 4.85 of 5 – based on 32 votes