Experiment Setup Properties
The XAutoCustomer experiment shell expects a single argument when
you start it. This is the name of a property file that controls the
setup parameters for a single experimental run. The format of the
file is a set of property_name = property_value pairs.
Comment lines start with the hash mark #. The following properties
are supported. Unless stated otherwise, all of them are read from the
named property file.
Run
- ExperimentId = string
- The string will be used as an ID string in all output files
for this experiment.
- LogConfiguration = filename
- The name of the setup file for the log4j logging system. The
default value is client.lcf. This system gives much
finer control of log output than the old Log system. In general,
we use Categories named for individual classes for trace output,
and a Category named "Result" for measurement data collection.
Note that if the ExperimentId property is set, the
filenames for trace and result output will be overridden in the
main method of XAutoCustomer.
- LogFile = filename
- deprecated The name of the file to write log info to.
Default is standard
out. NOTE: The LogFile system is swiftly being
edited out of the whole system, and very little useful information
is now emitted to it from the client code.
- LogLevel = int
- deprecated The level of detail to write to the Logfile.
Range is 0-4,
default is 3. Errors are level 1, detailed trace info is 4.
- MFile = filename
- deprecated Name of the file to write MatLab data out.
If this is missing, then no MatLab data will be written.
NOTE: as of 1/1/2001, most of the MatLab output
is disabled in the code.
- NRuns = int
- Number of runs to execute. All data from multiple runs will
be together in the log file and the MatLab file
- NSearches = int
- Number of times to run the search, after generating the plan
and acquiring bids. Default value is 1. It only makes sense to
run repeated searches with a
stochastic search, since presumably a deterministic method would
simply return the same results each time. It also doesn't make
much sense to use multiple searches in conjunction with a
non-default BidScheduler; in fact, the bid scheduler will be
disabled if the value of this parameter is greater than 1.
- PlanSeed = int
- Seed to use in generating random numbers in the bid-generation
code. Default is 100.
- BidSeed = int
- Seed to use in generating random numbers in the bid-generation
code. Default is 100.
- SearchSeed = int
- Seed to use in generating random numbers in the search
code. Default is 100.
- LogSeeds = boolean
- If "true" then at the beginning of each problem, all random
variable seeds will be incremented by a constant amount. Note
that the numbers generated under when this value is true will
be different than when they are false. This is because the
java.util.Randon.setSeed() operation is
non-reversible - you can't get the seed back out of a random
number. In other words, if you run a series of 4 problems
with LogSeeds not set or false, and then run a series with
LogSeeds true, they will be different problem sets.
Update: This feature does not work as well as
one would hope...
Plan Data
This data is taken from the main property file
- PlanSize = integer
- The number of subtasks to include in the plan.
- PlanType = { random | serial | parallel | mixed }
- Choice of plan generator to use. The AA-99 paper uses the
parallel, serial, and mixed plans, while the AAAI-99 paper uses
the random plans. (Question: Can this be extended
to include pre-generated named plans?) (Note: As
of early 2000, only random plans are supported.)
- PlanBranchFactor = double
- The average branching factor for plans created with the Random
plan generator. The actual number may end up being smaller than
the given factor, because most redundant precedence relations are
edited out after the plan is generated.
- TaskTypes = String
- Specifies the types of tasks used in generating plans, and the
relative frequency of each type. The format of the string is a
space-separated list of type:frequency pairs. Each type
named is expected to be described in a separate property file in
the current directory named type.props. See the Task Data
section below for format. If more than one type is specified,
then the entire string must be enclosed in double quotes.
Example:
TaskTypes = "short:3.0 long:1.0"
specifies that of every 4 tasks generated, on average 3 should be
short, and 1 long. In addition, there are two files,
short.props and long.props, from which detailed
task type specifications will be taken.
Task data
This section describes the contents of a Task Data property file.
- name = String
- The name for this task type. This String will be used as a
prefix for task names in the trace output.
- description = String
- A brief (1 line) description of the task type. It's not used
for anything at the moment.
- expected-duration = double
- The average bid duration, in hours, for tasks of this type. The actual
duration in a particular bid is chosen from a normal variable with
mean expected-duration and standard deviation
duration-var.
- duration-var = double
- The standard deviation of the bid duration value will be
duration-var times expected duration.
- expected-price = double
- The average price for tasks of this type. The actual
price in a particular bid is chosen from a normal variable with
mean expected-price and standard deviation
price-var.
- price-var = double
- The standard deviation of the bid price.
- supplier-availability = double
- The probability that a given supplier will attempt to generate
a bid for a task of this type.
- resource-availability = double
- The probability that any particular bidder will have resources
available for a task of this type at the requested time. A value
of 1.0 indicates that the supplier will definitely be able to bid
for a task within a time window no larger than the supplier's
duration, as determined by the expected-duration and
duration-var parameters. Otherwise, the size of the
required time window is on average
(1/resource-availability) times the supplier's duration
for the task. If that doesn't fit into the customer's time
window, then no bid will be submitted.
Bid Manager setup
This section appears in the main property file.
- BidCycle.BidDeadline = long
- Interval in milliseconds between bid submission and bid
deadline, scaled by AgentTime. In other words, for an AgentTime
rate value of 4.0, a BidDeadline of 40000 will result in a
10-second bid deadline interval.
- BidCycle.EarlyConsider = long
- Interval in milliseconds between the bid deadline and the time
bids are available, scaled by the AgentTime clock rate. This must
be somewhat longer than 1 second when the remote server is in use.
- BidCycle.EarlyOffer = long
- Interval in milliseconds between the Early Consider time and
the earliest time awards can be issued, scaled by the AgentTime
clock rate.
- BidScheduler.Type = String
- The unqualified class name of the desired bid scheduler.
Default value is "BidScheduler". These things are responsible for
deciding how to run the bidding process from the customer
side. (Programmer's Note: These classes must
reside in the package edu.umn.magnet.client.bid).
Other options include:
- "SchedFwdBySteps" divides the plan into a set of
approximately equal steps. The parameter
"BidScheduler.nsteps" determines the number of steps.
- "SchedFwdTilRisky" (not implemented yet)
- "SchedRiskyFirst" (not implemented yet)
- PlanSlack = double
- The ratio of allowed time to minimum expected time for the
plan schedule. A value of 1.2 gives a 20% overall slack to the
plan.
- PlanTWA.Type = String
- The unqualified class name of the class used to adjust time
windows in the RFQ. The default value is
"CPMTWAdjuster". (Programmer's Note:
These classes must reside in the package
edu.umn.magnet.client.bid). Available classes include:
- "CPMTWAdjuster" uses the CPM algorithm in
conjunction with the selected TDA to adjust time windows, after
expanding the overall plan by the value of "CPMSlack".
- "MinOverlapTWAdjuster" operates in three steps:
first the overall plan is expanded by the value of "CPMSlack"
(or the PlanDeadline is used, once it's implemented), second the
task durations are adjusted by the value of "TDA.DurationMult",
and finally the time windows are set to minimize and fairly
distribute the remaining overlap.
- CPMSlack = double
- The ratio of expected duration to schedule window for each
task in the plan. This is used to add slack to the schedule as
expressed in the RFQ. (Note: This parameter is
deprecated in favor of the "TDA" paradigm as of
September 2000.)
- TDA.type = String
- The unqualified class name of the
TaskDurationAdjuster subclass that
will be used to adjust task durations in the process of setting
the RFQ schedule. The default value is
"TaskDurationAdjuster", which leaves task durations
untouched. (Programmer's Note: These
classes must reside in the package
edu.umn.magnet.client.bid). Supported values are:
- "ProportionalTDA" adjusts task durations by the value of the
property "TDA.DurationMult", a double. Note that this is the
inverse of the old "CPMSlack" property, which was known to lead
to some confusion. If this value is greater than PlanSlack, you
are asking for trouble. The default value is 0.8.
- "PropOfferedTDA" adjusts task durations as in
"ProportionalTDA", but only for tasks that are in the set
currently offered for bids.
Supplier Bid data
- Sup.Method = String
- The method used to generate bid sets. Available options include:
- "ClassicSM" attempts to generate the number of bids specified
in the "Sup.BidCount" parameter. Some attempts may fail, and
there is no guarantee that a set giving full task coverage will
be generated.
- "GuaranteedCoverageSM" works like "Classic", except that it
attempts to ensure that every individual task will have at least one
bid. It does not guarantee feasibility. The parameter
"Sup.GuaranteedCoverageNTrials" determines how persistent this
will be at covering tasks that were not covered by the "Classic"
approach. A value of 1 will just give one try per non-covered
task, which may fail if the available time window is not large
enough or if resource availability is too low. Higher values
may be used to make it more persistent.
- Sup.BidMethod = String
- The method used to generate individual bids. Available
options include:
- "ContiguousBidder" generates bid combinations that contain
contiguous sets of task bids, that is, bids for tasks that are
related by precedence relations. Temporal feasibility among
the SubtaskBids within such a Bid is guaranteed. The parameter
"Sup.FollowLinkProb" specifies the probability that a
predecessor/successor link will be followed in the process of
generating a bid, and "Sup.FollowLinkVar" specifies the
variability in this number.
- "RandomBidder" generates bid combinations with random sets of
subtask bids. The size of each bid is chosen as
(bs + (r * bs * bv))
where bs is "Sup.BidSize", bv is "Sup.BidSizeVariability", and r
is a normally-distributed random value.
- Sup.BidCount = integer
- The number of bids the AutoSupplier will attempt to
generate.
- Sup.Bids/Task = double
- As an alternative to generating a fixed number of bids,
specifying this parameter will cause the SupplyMonster to continue
to generate bids until the ratio of subtask bids to tasks in the
incoming plan matches this parameter. Use of this parameter by
itself does not guarantee coverage.
- Sup.MaxDiscountPercent = double
- Sets the maximum percentage discount for the bids over the
combined price of the subtask bids in that bid.
Agent status management
- RFQStatus.Bids.DontWait
- If this is set to true, then the bid collection process
terminates as soon as the (single) batch of bids is returned,
short-circuiting the wait for the bid submission deadline.
- AgentTime.Base
- The "base time" for the AgentTime clock. If all components of
a distributed MAGNET system are set up with the same base, offset,
and rate values, and if their system clocks are synchronized by
NTP or some similar mechanism, then all components will agree on the
simulated time. The current AgentTime is given by
Base + Offset + (Rate * (System - Base))
The time format is "mm/dd/yyyy hh:mm" in 24-hour format. The time
zone is taken from the locale.
- AgentTime.Offset
- The offset value for AgentTime, as an integer number of
milliseconds.
- AgentTime.Rate
- The rate at which time passes in the Agent as a multiple of
"normal" speed. In other words, with a value of 60.0, an hour of
agent time would pass in a minute of "wall clock" time.
Evaluation Parameters
- CoveragePenalty = double
- Evaluation penalty for non-covered task in bid evaluation.
- FeasibilityPenalty = double
- Evaluation penalty for 1 minute of task overlap in bid
evaluation.
- DeadlinePenalty = double
- Amount to penalize tasks for coming in late or reward them for
coming in early. The penalty or reward per task can be nearly as
large as this parameter.
Search Control
Most of this is for controlling the Simulated Annealing search. It is
taken from the main properties file.
- Search = String
- The type of search to be run. Current legal values are:
- SA
- for the Simulated Annealing engine,
- systematic
- for the systematic search engine,
- IP
- for the Integer Programming engine using the "naive"
problem encoding, and
- IPCon
- for the Integer Programming engine using the
constraint-based problem encoding.
- Astar
- for the Bidtree-based A* search engine
- IDAstar
- for the Bidtree-based Iterative-Deepening A* search engine
- IP.solver = string
- The path to the image ip_solve. Default value is
"../../bin/ip_solve".
- IP.epsilon = double
- The error bound for the IP solver. Default value is 1e-5.
You shouldn't have to change this value.
- BTS.SortDirection = ["Increasing" |
"Decreasing"]
- Determines the sort order used when constructing the
bidtree. "Increasing" puts the tasks with the fewest bids first,
creating a narrow-first tree. "Decreasing" puts the tasks with the
most bids first, creating a wide-first tree. Performance is
generally, but not always, better with a "Decreasing" bidtree.
- BTS.MaxQueueSize = integer
- Limits the queue size of Bidtree searches (the A* search in
particular, since IDA* doesn't use the queue). If the queue
exceeds this size, it will be trimmed, and you are no longer
guaranteed an optimal result.
- BTS.UseSortedBids = ["True" | "False"]
- If True (the default value), then expansions of a given node
are done using the lowest-average-cost bid first, then the
next-lowest cost, etc. If False, then expansions are done "by
bucket", in which all bids that specify the same task set are
tried in sequence. If the first bid in a bucket fails a coverage
test, then the entire bucket is discarded and expansions continue
with the next bucket, if any. Bids within a bucket are sorted by
cost.
- IDA.Multiplier = float
- The amount to increase the evaluation bound for each cycle in
the IDA* solver. The default value is 1.05, but with the standard
test setup better results are obtained with a value in the range
of 1.15.
- SA.PatienceFactor = int
- This number, multiplied by the sum of the logs of the number
of subtasks and the number of bids, will control search
termination. When this number of nodes have been generated
without improvement, the search stops.
- PatienceFactor = int deprecated
- Older name for SA.PatienceFactor.
- SA.InitialTemperature = double
- This number, multiplied by the range of evaluations in the
search queue, added to the evaluation of the best node in the
queue, gives the initial average target evaluation for simulated
annealing node selection. The default value is 0.1.
- InitialTemperature = double
deprecated
- Obsolete name for SA.InitialTemperature.
- AnnealingRate = int deprecated
- The number of nodes to count off before applying the annealing
ratio to the temperature. The default value is 100.
- SA.AnnealingScaleFactor = double
- When multiplied by SA.PatienceFactor, gives a value for
AnnealingRate, overriding that value if also given. The
default value is 0.2.
- SA.AnnealingRatio = double
- A multiplier that is applied to the Temperature every time
AnnealingRate additional nodes have been generated. The default
value is 0.8.
- AnnealingRatio = double
deprecated
- Synonym for SA.AnnealingRatio, except that the value
of SA.AnnealingRatio overrides this value if both are
present.
- SA.StopOnSolution
- If true (default), stop the Simulated Annealing search at the
end of the first restart cycle that produces a solution, or at the
end of the nth cycle after that, where n
is the value of the ExtraCycles property.
- BeamWidth = int
- The number of nodes to retain on the search queue. When a
node is added that would exceed this limit, the node with the
worst evaluation is dropped.
The default value is 100.
- BidSelector = classname
- Choose the type of BidSelector to be used for the search.
Acceptable values are the unqualified class names of the subclasses of
edu.umn.magnet.client.eval.BidSelector. All of these classes must
be in the package edu.umn.magnet.client.eval. The default value
is "RandomBidSelector".
- SubtaskBidRatio = double
- A value between 0.0 and 1.0 that determines the ratio of
subtask bids to bids when using the random bid, random subtask bid
selector. The default is 0.5.
- TestUniqueness = true/false
- If true, then the search will not enqueue nodes that have
already been encountered, at the cost of some extra computational
effort. In general, this test reduces search effort and improves
the probability of finding solutions.
- ExcludeInfeasibleNodes = true/false
- If true, then the search will not enqueue infeasible nodes, on
the theory that they are less likely to lead to a good solutions.
- RecycleCount = int
- The number of times to restart the simulated annealing search
as long as no solution has been found. Typical values are 5-10.
- RecycleRatio = double
- The amount to adjust the patience factor on each restart. A
value of 1.0 says leave it alone. A typical value is 1.1.
- ExtraCycles = int
- The number of extra restarts to run the search after a
solution is found. This has been shown to slightly improve
solution quality at the cost of significant additional search effort.
- RestartUnique = true/false
- If true, the list of unique nodes is cleared on each restart.
Otherwise, each restart will be prevented from using nodes already
generated in previous searches. It's not clear how helpful this
is.
- LockAllBids = true/false
- If true, then expansion of TaskBidPlan nodes cannot involve
displacing a bid that is already assigned. This works for both
the Systematic and Simulated Annealing search engine.
- XA.Tolerance = double
- When running repeated search test, this is the "tolerance"
stopping condition. The searches will be aborted when the
evaluation is within 1 + XA.tolerance of the target
value. The default value is 0.0001.
Copyright: © 1999, 2000, 2001 by the Regents of the University
of Minnesota
Department of Computer Science and
Engineering. All rights reserved.
The University of Minnesota is an equal opportunity educator and
employer.
$Id: experiment_setup.html,v 1.23.2.1 2002/04/22 19:04:35 jcollins Exp $