Experiment Setup Properties

The XAutoCustomer experiment shell expects a single argument when you start it. This is the name of a property file that controls the setup parameters for a single experimental run. The format of the file is a set of property_name = property_value pairs. Comment lines start with the hash mark #. The following properties are supported. Unless stated otherwise, all of them are read from the named property file.

Run

ExperimentId = string
The string will be used as an ID string in all output files for this experiment.

LogConfiguration = filename
The name of the setup file for the log4j logging system. The default value is client.lcf. This system gives much finer control of log output than the old Log system. In general, we use Categories named for individual classes for trace output, and a Category named "Result" for measurement data collection. Note that if the ExperimentId property is set, the filenames for trace and result output will be overridden in the main method of XAutoCustomer.

LogFile = filename
deprecated The name of the file to write log info to. Default is standard out. NOTE: The LogFile system is swiftly being edited out of the whole system, and very little useful information is now emitted to it from the client code.

LogLevel = int
deprecated The level of detail to write to the Logfile. Range is 0-4, default is 3. Errors are level 1, detailed trace info is 4.

MFile = filename
deprecated Name of the file to write MatLab data out. If this is missing, then no MatLab data will be written. NOTE: as of 1/1/2001, most of the MatLab output is disabled in the code.

NRuns = int
Number of runs to execute. All data from multiple runs will be together in the log file and the MatLab file

NSearches = int
Number of times to run the search, after generating the plan and acquiring bids. Default value is 1. It only makes sense to run repeated searches with a stochastic search, since presumably a deterministic method would simply return the same results each time. It also doesn't make much sense to use multiple searches in conjunction with a non-default BidScheduler; in fact, the bid scheduler will be disabled if the value of this parameter is greater than 1.

PlanSeed = int
Seed to use in generating random numbers in the bid-generation code. Default is 100.

BidSeed = int
Seed to use in generating random numbers in the bid-generation code. Default is 100.

SearchSeed = int
Seed to use in generating random numbers in the search code. Default is 100.

LogSeeds = boolean
If "true" then at the beginning of each problem, all random variable seeds will be incremented by a constant amount. Note that the numbers generated under when this value is true will be different than when they are false. This is because the java.util.Randon.setSeed() operation is non-reversible - you can't get the seed back out of a random number. In other words, if you run a series of 4 problems with LogSeeds not set or false, and then run a series with LogSeeds true, they will be different problem sets.

Update: This feature does not work as well as one would hope...

Plan Data

This data is taken from the main property file

PlanSize = integer
The number of subtasks to include in the plan.

PlanType = { random | serial | parallel | mixed }
Choice of plan generator to use. The AA-99 paper uses the parallel, serial, and mixed plans, while the AAAI-99 paper uses the random plans. (Question: Can this be extended to include pre-generated named plans?) (Note: As of early 2000, only random plans are supported.)

PlanBranchFactor = double
The average branching factor for plans created with the Random plan generator. The actual number may end up being smaller than the given factor, because most redundant precedence relations are edited out after the plan is generated.

TaskTypes = String
Specifies the types of tasks used in generating plans, and the relative frequency of each type. The format of the string is a space-separated list of type:frequency pairs. Each type named is expected to be described in a separate property file in the current directory named type.props. See the Task Data section below for format. If more than one type is specified, then the entire string must be enclosed in double quotes. Example:
TaskTypes = "short:3.0 long:1.0"
specifies that of every 4 tasks generated, on average 3 should be short, and 1 long. In addition, there are two files, short.props and long.props, from which detailed task type specifications will be taken.

Task data

This section describes the contents of a Task Data property file.

name = String
The name for this task type. This String will be used as a prefix for task names in the trace output.

description = String
A brief (1 line) description of the task type. It's not used for anything at the moment.

expected-duration = double
The average bid duration, in hours, for tasks of this type. The actual duration in a particular bid is chosen from a normal variable with mean expected-duration and standard deviation duration-var.

duration-var = double
The standard deviation of the bid duration value will be duration-var times expected duration.

expected-price = double
The average price for tasks of this type. The actual price in a particular bid is chosen from a normal variable with mean expected-price and standard deviation price-var.

price-var = double
The standard deviation of the bid price.

supplier-availability = double
The probability that a given supplier will attempt to generate a bid for a task of this type.

resource-availability = double
The probability that any particular bidder will have resources available for a task of this type at the requested time. A value of 1.0 indicates that the supplier will definitely be able to bid for a task within a time window no larger than the supplier's duration, as determined by the expected-duration and duration-var parameters. Otherwise, the size of the required time window is on average (1/resource-availability) times the supplier's duration for the task. If that doesn't fit into the customer's time window, then no bid will be submitted.

Bid Manager setup

This section appears in the main property file.

BidCycle.BidDeadline = long
Interval in milliseconds between bid submission and bid deadline, scaled by AgentTime. In other words, for an AgentTime rate value of 4.0, a BidDeadline of 40000 will result in a 10-second bid deadline interval.

BidCycle.EarlyConsider = long
Interval in milliseconds between the bid deadline and the time bids are available, scaled by the AgentTime clock rate. This must be somewhat longer than 1 second when the remote server is in use.

BidCycle.EarlyOffer = long
Interval in milliseconds between the Early Consider time and the earliest time awards can be issued, scaled by the AgentTime clock rate.

BidScheduler.Type = String
The unqualified class name of the desired bid scheduler. Default value is "BidScheduler". These things are responsible for deciding how to run the bidding process from the customer side. (Programmer's Note: These classes must reside in the package edu.umn.magnet.client.bid). Other options include:

PlanSlack = double
The ratio of allowed time to minimum expected time for the plan schedule. A value of 1.2 gives a 20% overall slack to the plan.

PlanTWA.Type = String
The unqualified class name of the class used to adjust time windows in the RFQ. The default value is "CPMTWAdjuster". (Programmer's Note: These classes must reside in the package edu.umn.magnet.client.bid). Available classes include:

CPMSlack = double
The ratio of expected duration to schedule window for each task in the plan. This is used to add slack to the schedule as expressed in the RFQ. (Note: This parameter is deprecated in favor of the "TDA" paradigm as of September 2000.)

TDA.type = String
The unqualified class name of the TaskDurationAdjuster subclass that will be used to adjust task durations in the process of setting the RFQ schedule. The default value is "TaskDurationAdjuster", which leaves task durations untouched. (Programmer's Note: These classes must reside in the package edu.umn.magnet.client.bid). Supported values are:

Supplier Bid data

Sup.Method = String
The method used to generate bid sets. Available options include:

Sup.BidMethod = String
The method used to generate individual bids. Available options include:

Sup.BidCount = integer
The number of bids the AutoSupplier will attempt to generate.

Sup.Bids/Task = double
As an alternative to generating a fixed number of bids, specifying this parameter will cause the SupplyMonster to continue to generate bids until the ratio of subtask bids to tasks in the incoming plan matches this parameter. Use of this parameter by itself does not guarantee coverage.

Sup.MaxDiscountPercent = double
Sets the maximum percentage discount for the bids over the combined price of the subtask bids in that bid.

Agent status management

RFQStatus.Bids.DontWait
If this is set to true, then the bid collection process terminates as soon as the (single) batch of bids is returned, short-circuiting the wait for the bid submission deadline.

AgentTime.Base
The "base time" for the AgentTime clock. If all components of a distributed MAGNET system are set up with the same base, offset, and rate values, and if their system clocks are synchronized by NTP or some similar mechanism, then all components will agree on the simulated time. The current AgentTime is given by
    Base + Offset + (Rate * (System - Base))
    
The time format is "mm/dd/yyyy hh:mm" in 24-hour format. The time zone is taken from the locale.

AgentTime.Offset
The offset value for AgentTime, as an integer number of milliseconds.

AgentTime.Rate
The rate at which time passes in the Agent as a multiple of "normal" speed. In other words, with a value of 60.0, an hour of agent time would pass in a minute of "wall clock" time.

Evaluation Parameters

CoveragePenalty = double
Evaluation penalty for non-covered task in bid evaluation.

FeasibilityPenalty = double
Evaluation penalty for 1 minute of task overlap in bid evaluation.

DeadlinePenalty = double
Amount to penalize tasks for coming in late or reward them for coming in early. The penalty or reward per task can be nearly as large as this parameter.

Search Control

Most of this is for controlling the Simulated Annealing search. It is taken from the main properties file.

Search = String
The type of search to be run. Current legal values are:
SA
for the Simulated Annealing engine,
systematic
for the systematic search engine,
IP
for the Integer Programming engine using the "naive" problem encoding, and
IPCon
for the Integer Programming engine using the constraint-based problem encoding.
Astar
for the Bidtree-based A* search engine
IDAstar
for the Bidtree-based Iterative-Deepening A* search engine

IP.solver = string
The path to the image ip_solve. Default value is "../../bin/ip_solve".

IP.epsilon = double
The error bound for the IP solver. Default value is 1e-5. You shouldn't have to change this value.

BTS.SortDirection = ["Increasing" | "Decreasing"]
Determines the sort order used when constructing the bidtree. "Increasing" puts the tasks with the fewest bids first, creating a narrow-first tree. "Decreasing" puts the tasks with the most bids first, creating a wide-first tree. Performance is generally, but not always, better with a "Decreasing" bidtree.

BTS.MaxQueueSize = integer
Limits the queue size of Bidtree searches (the A* search in particular, since IDA* doesn't use the queue). If the queue exceeds this size, it will be trimmed, and you are no longer guaranteed an optimal result.

BTS.UseSortedBids = ["True" | "False"]
If True (the default value), then expansions of a given node are done using the lowest-average-cost bid first, then the next-lowest cost, etc. If False, then expansions are done "by bucket", in which all bids that specify the same task set are tried in sequence. If the first bid in a bucket fails a coverage test, then the entire bucket is discarded and expansions continue with the next bucket, if any. Bids within a bucket are sorted by cost.

IDA.Multiplier = float
The amount to increase the evaluation bound for each cycle in the IDA* solver. The default value is 1.05, but with the standard test setup better results are obtained with a value in the range of 1.15.

SA.PatienceFactor = int
This number, multiplied by the sum of the logs of the number of subtasks and the number of bids, will control search termination. When this number of nodes have been generated without improvement, the search stops.

PatienceFactor = int deprecated
Older name for SA.PatienceFactor.

SA.InitialTemperature = double
This number, multiplied by the range of evaluations in the search queue, added to the evaluation of the best node in the queue, gives the initial average target evaluation for simulated annealing node selection. The default value is 0.1.

InitialTemperature = double deprecated
Obsolete name for SA.InitialTemperature.

AnnealingRate = int deprecated
The number of nodes to count off before applying the annealing ratio to the temperature. The default value is 100.

SA.AnnealingScaleFactor = double
When multiplied by SA.PatienceFactor, gives a value for AnnealingRate, overriding that value if also given. The default value is 0.2.

SA.AnnealingRatio = double
A multiplier that is applied to the Temperature every time AnnealingRate additional nodes have been generated. The default value is 0.8.

AnnealingRatio = double deprecated
Synonym for SA.AnnealingRatio, except that the value of SA.AnnealingRatio overrides this value if both are present.

SA.StopOnSolution
If true (default), stop the Simulated Annealing search at the end of the first restart cycle that produces a solution, or at the end of the nth cycle after that, where n is the value of the ExtraCycles property.

BeamWidth = int
The number of nodes to retain on the search queue. When a node is added that would exceed this limit, the node with the worst evaluation is dropped. The default value is 100.

BidSelector = classname
Choose the type of BidSelector to be used for the search. Acceptable values are the unqualified class names of the subclasses of edu.umn.magnet.client.eval.BidSelector. All of these classes must be in the package edu.umn.magnet.client.eval. The default value is "RandomBidSelector".

SubtaskBidRatio = double
A value between 0.0 and 1.0 that determines the ratio of subtask bids to bids when using the random bid, random subtask bid selector. The default is 0.5.

TestUniqueness = true/false
If true, then the search will not enqueue nodes that have already been encountered, at the cost of some extra computational effort. In general, this test reduces search effort and improves the probability of finding solutions.

ExcludeInfeasibleNodes = true/false
If true, then the search will not enqueue infeasible nodes, on the theory that they are less likely to lead to a good solutions.

RecycleCount = int
The number of times to restart the simulated annealing search as long as no solution has been found. Typical values are 5-10.

RecycleRatio = double
The amount to adjust the patience factor on each restart. A value of 1.0 says leave it alone. A typical value is 1.1.

ExtraCycles = int
The number of extra restarts to run the search after a solution is found. This has been shown to slightly improve solution quality at the cost of significant additional search effort.

RestartUnique = true/false
If true, the list of unique nodes is cleared on each restart. Otherwise, each restart will be prevented from using nodes already generated in previous searches. It's not clear how helpful this is.

LockAllBids = true/false
If true, then expansion of TaskBidPlan nodes cannot involve displacing a bid that is already assigned. This works for both the Systematic and Simulated Annealing search engine.

XA.Tolerance = double
When running repeated search test, this is the "tolerance" stopping condition. The searches will be aborted when the evaluation is within 1 + XA.tolerance of the target value. The default value is 0.0001.


Copyright: © 1999, 2000, 2001 by the Regents of the University of Minnesota
Department of Computer Science and Engineering. All rights reserved.
The University of Minnesota is an equal opportunity educator and employer.

$Id: experiment_setup.html,v 1.23.2.1 2002/04/22 19:04:35 jcollins Exp $