10.4 Development Release Series 8.5

This is the development release series of HTCondor. The details of each version are described below.

Version 8.5.8

Release Notes:

• HTCondor version 8.5.8 released on December 13, 2016.

New Features:

• On Linux, the starter now puts all jobs in a cgroup by default. The default for CGROUP_MEMORY_LIMIT_POLICY is now "none". To disable cgroups, an admin can set the BASE_CGROUP parameter to the empty string. (Ticket #5936).

• Added first-class condor_submit commands supporting job retries. (See section 11 for details.) (Ticket #5912).

• condor_qedit now defaults to editing only jobs owned by the current user in the same way that condor_q does. It also honors the CONDOR_Q_ONLY_MY_JOBS configuration variable. (Ticket #5889).

• Added new parameter DOCKER_VOLUME_DIR_XXX_MOUNT_IF which is an expression, evaluated in the context of the machine and job ad, which if it evaluates to a string, becomes a docker volume mount. This allows admins to conditionally add docker volumes for certain types of jobs. (Ticket #5758).

• Added initial support for Singularity containers. (Ticket #5828).

• The XferStatsLog file on the submit side now contains TCP statistics for both the shadow point of view, and the starter point of view. The starter side line is prefixed with the words "peer stats from starter". (Ticket #5917).

• Configuration variables of the form SUBSYS.LOCALNAME.VARIABLE no longer work. The use of the SUBSYS prefix before LOCALNAME never worked fully, and was only necessary for while as a workaround for a bug that was fixed many years ago. condor_config_val and the condor_master will now produce warning messages when the configuration has variables that appear to of this form and begin with a known SUBSYS name like MASTER or COLLECTOR. (Ticket #5969).

• The SLOT_WEIGHT parameter can now be set on the central manager, instead of all the execute nodes. If the execute nodes set this parameter, it will override the central manager setting. (Ticket #5953).

• New submit command gce_json_file can be used with grid-type gce jobs to specify a file that contains JSON object members that should be added to the instance description submitted to the GCE service. (Ticket #5893).

• A number of command-line tools now support bash auto-completion. (Ticket #5924).

• The minimum update time for condor_dagman node status files now defaults to 60 seconds. (Ticket #5929).

• Added the new DAGMAN_REMOVE_NODE_JOBS configuration macro, which allows users to configure whether condor_dagman itself removes its node jobs when it is removed (note that the node jobs are also removed by the condor_schedd). This configuration macro defaults to True, which represents a change in behavior compared to previous HTCondor versions. (See section 2.10.7 for more details.) (Ticket #5175).

• The -AllowLogError argument to condor_submit_dag and condor_dagman, and the DAGMAN_ALLOW_LOG_ERROR configuration macro, are no longer supported, and generate warnings if used. (Ticket #5630).

• condor_dagman now ignores the DAGMAN_LOG_ON_NFS_IS_ERROR configuration setting if ENABLE_USERLOG_LOCKING is set to False. (Ticket #5641).

• Added the ALL_NODES option to a number of condor_dagman commands (see 2.10.9 for details). (Ticket #5729).

• Changed the previous term "metaknob" to "configuration template" and improved the configuration template documentation. (Ticket #5865).

• The condor_schedd receiving a refreshed X.509 proxy credential is now done in a non-blocking fashion. (Ticket #5930).

• The Job Router now performs its automatic job ad transformations when the TRANSLATE_JOB hook is used. These are changes that should happen to all job ads being transformed by the Job Router. (Ticket #5235).

• The $F() configuration macro has new options to support conversions of paths to Windows style path separators or to Unix style. When used in condor_submit files it can do path completion as well. (Ticket #5938).

• The $ENV() configuration macro now supports default values. (Ticket #5882).

• A certificate mapfile can now use literal values rather than regular expressions for the second field. This is useful when only a single identity should be matched. The use of a literal is both more secure and faster to search. The new configuration variable CERTIFICATE_MAPFILE_ASSUME_HASH_KEYS enables this behavior, it defaults to false. It will most likely default to true in a future version of HTCondor. (Ticket #5992).

• The ClassAd userMap function now uses only commas as the separator for the third field of the map file. This makes it possible to have values with spaces in them. (Ticket #5988).

• The condor_collector will now allow more than one condor_negotiator to be registered. And a new A new configuration variable COLLECTOR_ALLOW_ONLY_ONE_NEGOTIATOR, which defaults to false has been added so that the old behavior can still be configured. (Ticket #5967).

• The Requirements expression for Job transforms in the condor_schedd will now ignore the TARGET prefix for attributes in the expression. This makes it easier to convert condor_job_router rules to job transforms because the TARGET prefix is required in the condor_job_router but refers to nothing in the job transform. (Ticket #5980).

• The -better-analyze option of condor_q has been improved and the output reorganized. (Ticket #5290).

• A new tool - condor_transform_ads has been added. (See section 11 for details.) (Ticket #5805).

• A join function has been added to the ClassAd language. (Ticket #6018).

• condor_who has additional options for querying the state and readiness of the various daemons. It has a command that can be used to wait for the daemons to startup with a timeout. (Ticket #5416).

• When submitting a job that has an associated X.509 proxy, or when authenticating to the condor_schedd using X.509, the X.509 and VOMS attributes are securely extracted and carried along in the job ClassAd. This allows them to be used, for example, in matchmaking policy and job routing. (Ticket #5064).

• Made condor_credd configuration easier by automatically configuring network connections to use encryption.

Bugs Fixed:

• When the Google Compute Engine breaks the results of a query into multiple pages, the gce_gahp now retrieves all of the results, instead of just the first page. (Ticket #6010).

• Fixed a bug that caused file transfer to fail when a job created by the Job Router has a different Owner than the original job. (Ticket #5348).

• Fixed a bug that could result in "orphan" node jobs staying in the queue when an instance of condor_dagman is removed. (Ticket #5702).

• Fixed a regression introduced in v8.5.7 that prevents job preemption due to priority from occurring, because user priority and resources in use information cannot be referenced in PREEMPTION_REQUIREMENTS. (Ticket #6014).

• Fixed COLLECTOR_FORWARD_FILTERING so that a startd ad update is always forwarded when any of the Claim IDs change. (Ticket #5913).

• Fixed a bug that made the Requirements keyword for job transforms in the condor_schedd only work if it was all uppercase on Red Hat 7 and some other platforms that use a newer version of the C++ compiler. (Ticket #5973).

• Fixed a bug that allowed a user to bypass the MAX_RUNNING_SCHEDULER_JOBS_PER_OWNER limit by specifying an accounting group or nice_user in their submit file. (Ticket #5949).

• Fixed a bug in condor_c and the condor_job_router that could cause inaccurate job totals to be reported by condor_q -batch. (Ticket #6020).

Version 8.5.7

Release Notes:

• HTCondor version 8.5.7 released on September 29, 2016.

Known Issues:

• Preemption due to job priority is likely to fail if PREEMPTION_REQUIREMENTS attempts to reference any resource usage or priority attributes. This issue has been fixed in v8.5.8. If you cannot upgrade to v8.5.8, a work-around for v8.5.7 is to set configuration macro NEGOTIATOR_CROSS_SLOT_PERI'S to True. (Ticket #6014).

New Features:

• Added the capability for the schedd to perform job ClassAd transformations upon job submission (see 3.7.2 for details). (Ticket #5885).

• Added the capability for more flexible connections between splices in DAGs (see 2.10.9 for details). Also added an INCLUDE command to the DAG language (see  2.10.9 for details). (Ticket #5213).

• Simplified the DAG node priority algorithm: the "effective" priority of a node is now simply the sum of the explicit node priority and the overall DAG priority. (See section 2.10.9 for more details.) (Ticket #4024). (Ticket #5749).

• Allow the second argument of the ClassAd ternary operator (expression ? value1 : value2) to be omitted. This new syntax means: evaluate the expression, and if it evaluated to a defined value or error, return it. If undefined, return value2. (Ticket #5782).

• The time is now included after the SCHEDD or SUBMITTER name in the banner of condor_q output. (Ticket #5895).

• condor_status has a new -data option that, when used with -schedd will show data transfer information; and -run will show information about running jobs when used with -schedd. (Ticket #3938).

• condor_q -batch will now show Total and Completed counts for non-DAG jobs when querying a scheduler that is at least version 8.5.7 (Ticket #5874).

• condor_status and condor_q now support reading and writing ClassAds in xml, json, and "new ClassAd" form as well as the traditional long form. (Ticket #5844). (Ticket #5820).

• HTCondor daemons now respect <LOCALNAME>.<SUBSYSTEM>_LOG if passed a -local-name parameter, and default to using $(LOG)/<Localname>Log if the former is not set. (Ticket #5768).

• HTCondor now automatically passes the -local-name parameter to a DC daemon if its entry in the DAEMON_LIST is not in the default DC_DAEMON_LIST. This should result in simpler and less error-prone configuration. (Ticket #5768).

• HTCondor now detects if an entry in DAEMON_LIST shares a binary with an entry in DC_DAEMON_LIST and marks the former as a DC daemon if so. This should result in simpler and less error-prone configuration. (Ticket #5767).

• Increase the resolution of file transfer timing statistics in the XferStatsLog to hundreds of a second. (Ticket #5898).

• The default host based security meta-knob now works in IPv6 only networks out of the box. (Ticket #5894).

• Old HAD configurations, with or without replication, should now work by default (without shared port). (Ticket #5769).

• HTCondor no longer gives up if a bad networking configuration is detected while running a tool. This allows condor_config_val to be used to debug the problem. (Ticket #5532).

• The condor_negotiator by default no longer cross advertises the user priority and resources in use from every slot in a machine ad to every other slot in that machine ad. NEGOTIATOR_CROSS_SLOT_PRIOS = true re-enables the old behavior. The accounting information for the current user of the slot remains advertised. (Ticket #5785).

• New submit attribute gce_preemptible allows the creation of preemptible Google Compute Engine (GCE) instances. These instances have a lower price, but can be interrupted at any time. Also added support for service accounts with GCE. (Ticket #5821).

• When submitting jobs to Slurm via the grid universe, the Slurm partition can now be specified using the batch_queue submit command. (Ticket #5780).

• Some old STARTD policy helper configuration variables were moved into two new configuration templates - FEATURE : UWCS_DESKTOP_POLICY_VALUES and FEATURE : TESTINGMODE_POLICY_VALUES (Ticket #5871).

• condor_submit on Windows will no longer insert the OSVERSIONINFO fields like WindowsMajorVersion into each job automatically. This is controlled by a new configuration variable SUBMIT_PUBLISH_WINDOWS_OSVERSIONINFO which defaults to false. (Ticket #5873).

• Added the option to cache the output of commands used in configuration files, so that the command doesn't have to be re-run every time the configuration file is referenced. Also added error and warning keywords to allow configuration files to report errors and warnings. (Ticket #5781).

Bugs Fixed:

• Fixed a bug in how the HAD daemon checks to see if it and its corresponding replication daemon were configured to be on the same host. (Ticket #5849).

• The EC2 GAHP now handles integer overflows when checking deadlines. This prevents spurious time-outs on 32-bit systems which have been up for more than 28 days. (Ticket #5824).

• Lengthened the watchdog timeout in the systemd service file to 20 minutes. Also, ping systemd at a third of the watchdog interval. (Ticket #5837).

• Fixed a bug that could cause daemons to create a file named dprintf_failure.SUBSYS if they failed to find the mail program. (Ticket #5854).

• For grid-type batch jobs, improved handling of command line arguments and environment variables that contain characters that have meaning to the shell. Previously, the presence of these characters would cause job execution to fail. (Ticket #5747).

• Fixed a bug that caused condor_config_val to segfault when the -name argument was used and the machine did not exist (Ticket #5818).

• Fixed a bug that caused condor_q -autocluster to crash unless the -nobatch option was also used. (Ticket #5839).

• Fixed a bug in the Python bindings where a thread executed python byte code without holding the global interpreter lock. (Ticket #5864).

Version 8.5.6

Release Notes:

• HTCondor version 8.5.6 released on August 2, 2016.

New Features:

• The default output of condor_q is now the -batch output. To change the default back to its pre-8.5.6 value, set the new configuration variable CONDOR_Q_DASH_BATCH_IS_DEFAULT to False. (Ticket #5708).

• A new class - the Submit class - was added to the Python bindings. It allows for the submission of HTCondor jobs via the Python bindings using the same keywords and automatic behavior as condor_submit. See section 6.7.1 for details. (Ticket #5666).

• The ability to send condor_drain commands is now exposed through the Python bindings. See section 6.7.1 for details. (Ticket #5507).

• The value of the configuration parameter DOCKER_DROP_ALL_CAPABILITIES is now no longer just true or false, but a ClassAd expression evaluated in the context of the machine (my) and the job (target). (Ticket #5759).

• When running Docker Universe containers on docker version 1.11 and newer, HTCondor now also sets -no-new-privs, to prevent setuid and setgid programs from running in containers, unless DOCKER_DROP_ALL_CAPABILITIES evaluated to false. (Ticket #5680).

• The hostname of the container that Docker Universe jobs run in is now set to a more useful name. Instead of a hash, it now contains the job's owner, the cluster and proc of the job, and the hostname of the machine the container runs on. (Ticket #5760).

• New options have been added to condor_history, so that condor_history can be used as the the HISTORY_HELPER for remote condor_history. The options are:
  • -since Scanning of the history file stops when an expression becomes true or a job id is read.
  • -completedsince Scanning of the history file stops when a job completed earlier than this time is read.
  • -scanlimit Used by remote condor_history to limit the number of jobs read from the history file.
  • -attributes Used by remote condor_history to limit the attributes transferred back.
  • -inherit Used by remote condor_history to define the socket to write results to.
  • -stream-results Used by remote condor_history so that results can be printed as they arrive.
  (Ticket #5642).

• Condorhistory will default to doing a remote query if there is a SCHEDD_HOST configured. This behavior can be defeated by passing the new -local argument. (Ticket #5765).

• The high-availability and replication daemons may now use shared port. (Ticket #5726).

• ClassAds can now be represented in JSON format. condor_q, condor_status, and condor_history have a -json command line option, which causes their output to be printed in JSON. (Ticket #5688).

• condor_dagman now allows commands to be more flexibly ordered within a DAG file. (See section 2.10.3 for details.) (Ticket #5732).

• Any accounting_group and accounting_group_user values specified for a DAG are now propagated to all jobs of the workflow, including sub-DAGs. (Ticket #5077).

• A new configuration variable MAX_RUNNING_SCHEDULER_JOBS_PER_OWNER can be used to limit the number of DAGs that any single user can have running at a time. (Ticket #5568).

• Monitoring the status of PBS and Slurm jobs is now much more efficient. Now, one query to the batch system is done for all jobs, instead of a separate query for each job. (Ticket #5722).

• Simplified how job leases are handled for grid universe jobs. Now, all jobs going to the same remote resource share a single lease time. (Ticket #5625).

• Added several statistics about commands issued to the GAHP server to the grid ads that the condor_gridmanager sends to the condor_collector:
  • GahpCommandsIssued
  • GahpCommandsTimedOut
  • GahpCommandsInFlight
  • GahpCommandsQueued
  • GahpCommandRuntime
  (Ticket #5698).

• The condor_shadow, condor_starter and condor_c-gahp daemons now log TCP statistics for file transfers. See 3.5.2 for more details. (Ticket #5663).

• Job ads now include NumJobCompletions, which counts the number of times a job exited of its own accord (successfully or not) and then successfully completed file transfer (if any was requested). (Ticket #5705).

• Kerberos authentication is now non-blocking, allowing an HTCondor daemon authenticating clients with Kerberos to handle more simultaneous incoming connections. (Ticket #5737).

• Password authentication is now non-blocking, allowing an HTCondor daemon authenticating clients with the PASSWORD method to handle more simultaneous incoming connections. (Ticket #5602).

• The full path to the submit file is now available as an automatic submit variable. (Ticket #5677).

• A new function userMap() has been added to the ClassAd language to facilitate the mapping of users to groups in the condor_schedd and condor_job_router (see 4.1.2 for details). (Ticket #5751).

• Configuration files now support the declaration of multi-line values, the is primarily of use when configuring the condor_job_router. (Ticket #5721).

• Configuration templates can now take arguments. (Ticket #5739).

• Improved the performance of the condor_negotiator when running with a large number of users or groups. The accounting data is only written to disk when it changes, not unconditionally. (Ticket #5719).

Bugs Fixed:

• Fixed a bug in Docker universe that required the name of a transferred executable to begin with "./" (Ticket #5761).

• Fixed a bug the prevented Docker universe jobs from reporting their network usage correctly. (Ticket #5750).

• condor_who now reports docker universe jobs more completely. (Ticket #5740).

• Fixed bugs preventing HTCondor daemons from recognizing an address in Sinful format as its own when operating in mixed (IPv4 and IPv6) mode. One manifestation of this would be errors from the HAD daemon when specifying hosts by name in the HAD_LIST or REPLICATION_LIST. (Ticket #5728). (Ticket #5776).

• condor_user_prio now more correctly shows information about submitters flocking to a pool, but who haven't used any resources. (Ticket #5743).

• No longer leak a file in the user's home directory each time a job is submitted to Slurm. (Ticket #5742).

• Fix a bug that prevented HTCondor from removing jobs from Slurm. (Ticket #5804).

• Fixed a bug when attempting to authenticate using multiple methods wherein if a method failed, the remaining methods were not always attempted. (Ticket #5673).

• Fixed a bug that prevented the condor_schedd from reading the job's X.509 proxy file when writing information to the SCHEDD_AUDIT_LOG. (Ticket #5770).

• Fixed a bug in condor_q where the SIZE column would not grow as needed to fit the data. (Ticket #5667).

• Fixed a bug where the condor_schedd did not treat a user as a queue superuser when it should have if the configuration included a map file, which is common for GSI authentication. (Ticket #5530).

• Lengthen the watchdog timeout in the systemd service file to 1 minute. The previous value of 5 seconds has taken down HTCondor for a single slow DNS query. (Ticket #5819).

Version 8.5.5

Release Notes:

• HTCondor version 8.5.5 released on June 6, 2016.

New Features:

• The EC2 GAHP now rate-limits its requests, and responds to overload warnings with an exponential back-off. Additionally, fewer operations are now performed on a per-job basis (as few as one in some cases). The resulting scalability improvements have been demonstrated to permit a single GAHP to manage ten thousand instances. Because the overload condition is account- and region- specific, the grid manager now launches a GAHP for each account-region pair. We therefore recommend adding D_PID to EC2_GAHP_DEBUG, for disambiguation, and this is now the default. (Ticket #5561). (Ticket #5588). (Ticket #5620).

• The grid manager now assigns HoldReasonCodes and HoldReasonSubCodes to EC2 jobs when they go on hold. Values are subject to change until the stable release. (Ticket #5628).

• The grid manager now advertises some metrics from the EC2 GAHP. (Ticket #5580).

• Some Linux distributions for supercomputer compute nodes and others distributions for docker images have no /var/run/utmp. HTCondor no longer aborts when this file is missing, when it tries to determine keyboard idle times, it just assumes these kinds of machines have no keyboards. (Ticket #5624).

• Docker Universe jobs now correctly advertise RemoteUserCpu and RemoteSysCpu in their job ad and in the job log file. (Ticket #5609).

• A batch name specified for a DAG (with the condor_submit_dag -batch-name option) is now propagated to all jobs of that DAG, including sub-DAGs. (Ticket #5493).

• The batch name for a condor_dagman job (if not set) now defaults to DagFile+cluster (where DagFile is the primary DAG file of the condor_dagman job, and cluster is the HTCondor cluster of the condor_dagman job). Because the batch name is now propagated throughout a workflow, if no batch name is specified, the batch name for all jobs in the workflow will be DagFile+cluster of the top-level condor_dagman job. (Ticket #5605).

• The files named in the submit file attributes vm_disk, xen_kernel, and xen_initrd now refer to locations on the execute machine. condor_submit no longer modifies these values or checks for their existence on the submit machine. If these files need to be transferred by HTCondor, then they should be listed in transfer_input_files and their presence in these vm universe attributes shouldn't include any path information. (Ticket #4167).

• In the python bindings, an ExprTree can be cast to an integer or floating point value. (Ticket #5636).

• HTCondor now supports the following systemd features: Socket Activation, Watchdog, Status message, and journald logging. In these release, the Socket Activation is not configured, because the security system is not prepared to properly handle the socket passed in from outside HTCondor. (Ticket #4144).

• Added config knob DEFAULT_MASTER_SHUTDOWN_SCRIPT to specify a default program to exec as root upon condor_master exit. See Section 3.5.7 for details. (Ticket #5590).

• The python bindings now support a per-thread security context, allowing the modification of various parameters such as the pool password and the X509UserProxy location. (Ticket #5632).

Bugs Fixed:

• Fixed a bug that caused file transfers to fail when using Bosco. (Ticket #5704).

Version 8.5.4

Release Notes:

• HTCondor version 8.5.4 released on May 2, 2016.

• The deltacloud type in the grid universe, which allowed submission to Deltacloud services, has been removed. (Ticket #5569).

New Features:

• condor_status can now display the utilization of a condor_startd with a single line of output for each machine rather than a line per slot. In this release this output is enabled by passing -compact to condor_status but in a future release this will be the default output of condor_status. (Ticket #5596).

• Improved the performance of the condor_collector by not computing dropped update statistics, statistics which have never been accessible by users. (Ticket #5566).

• The performance of the condor_history tool has been significantly improved. (Ticket #5536).

• condor_user_prio now queries the condor_collector for accounting information by default, when appropriate. This should be much faster than the older way of querying the condor_negotiator. The old path is still available by passing the -negotiator option to the tool. (Ticket #5508)..

• The default value of DAGMAN_ALWAYS_RUN_POST has been changed from True to False. This means that, by default, if the PRE script of a DAG node fails, the POST script of the node will not be run. (This had been the default behavior until version 7.7.2. The 7.7.2-8.5.3 behavior can be restored by setting DAGMAN_ALWAYS_RUN_POST to True, or by passing the new -AlwaysRunPost argument to condor_submit_dag.) (Ticket #5477).

• The batch_gahp can now submit multi-core jobs to HTCondor. (Ticket #5638).

• The batch_gahp's ability to generate a limited X.509 proxy for use by the job on the execute machine can now be disabled, which is now the default. (Ticket #5601).

• The condor_schedd will now send submitter ad updates for idle submitters less frequently than updates for submitters that have jobs in the queue. There are two new configuration variables to control this behavior. ABSENT_SUBMITTER_LIFETIME is the number of seconds after the last job for that submitter leaves the queue that the submitter will continue to send updates to the condor_collector. It defaults to 1 week. ABSENT_SUBMITTER_UPDATE_RATE is the maximum rate in seconds at which the condor_schedd will send updates to the condor_collector for a submitter that has no jobs in the queue. It defaults to 5 minutes. (Ticket #5559).

Bugs Fixed:

• Fixed a bug that caused the condor_schedd to exit when receiving an updated X.509 proxy for a job. (Ticket #5645).

• In expressions in the Job Router's configuration, attributes no longer require a 'TARGET.' scope prefix. (Ticket #5550).

• Fixed a bug in condor_q -xml that would put the XML header after the body unless -stream was passed. (Ticket #5597).

Version 8.5.3

Release Notes:

• HTCondor version 8.5.3 released on March 24, 2016.

New Features:

• ENABLE_IPV4 and ENABLE_IPV6 both now accept the special value "AUTO", which is true if an interface with the corresponding protocol exists on the host, and false otherwise. (Ticket #5524).

• ENABLE_IPV4 and ENABLE_IPV6 both now default to the special value "AUTO". Additionally, the new configuration macro PREFER_IPV4 is true by default. This macro causes HTCondor to prefer IPv4 over IPv6 when choosing an address to advertise, when choosing the address of daemon looked up in the collector, and when resolving DNS queries. (Ticket #5104).

• New configuration macros added: IPV4_ADDRESS, IPV6_ADDRESS, IP_ADDRESS_IS_V6. (Ticket #5512).

• New attributes have been added to the Submitter ClassAd to indicate the number of Idle and Running jobs for Scheduler universe and for Local universe. (Ticket #5519).

• Jobs can now be submitted to the Slurm batch scheduling system via the new slurm type in the grid universe. (Ticket #5515).

• In addition to logging to the file KERNEL_TUNING_LOG, the default LINUX_KERNEL_TUNING_SCRIPT now also logs to syslog and /etc/systcl.d/99-htcondor.conf. (Ticket #5489).

• condor_history -autoformat now supports the j option to print job ids like condor_q does. (Ticket #5558).

• HTCondor is now built and linked with Globus 6.0. (Ticket #5520).

• Pre-size the ClassAd hash table to improve the performance of the condor_collector when getting ClassAd updates. (Ticket #5551).

• The negotiator now forwards accounting information to the collector, where it can be easily queried and monitored. (Ticket #5491).

Bugs Fixed:

• Fixed a bug on condor_history that could result in truncation of the job id field. (Ticket #5527).

Version 8.5.2

Release Notes:

• HTCondor version 8.5.2 released on February 18, 2015.

New Features:

• condor_q now defaults to showing only the current user's jobs, unless the current user is a queue superuser. This behavior can be turned off by setting the new configuration option CONDOR_Q_ONLY_MY_JOBS to False in the configuration for either the condor_schedd or condor_q. It can also be turned off by passing the new -allusers command line option to condor_q. (Ticket #5271).

• Added support for immutable job attributes and protected job attributes. (Ticket #5065).

• Improved the speed of condor_q and other tools when querying non-local servers. (Ticket #5150).

• The new -batch command line option to condor_q can be used to show a single line of progress information for a batch of jobs, where a batch is either an entire workflow (a DAG, including sub-DAGs), all of the jobs in a cluster, all of the jobs from a single user that have the same executable specified in their submit file, or all of the jobs from a single user that have the same batch name. (Ticket #4976).

• condor_submit and condor_submit_dag both have a new command line option -batch-name which can be used to set the batch name used by condor_q -batch. (Ticket #5490).

• condor_q and condor_status now attempt to avoid truncating data by making use of the full width of the terminal. (Ticket #5429). (Ticket #5459).

• Docker Universe jobs now report and update ResidentSetSize, ImageSize, MemoryUsage, NetworkInput, and NetworkOutput attributes in the job ad and log file. (Ticket #5456).

• Added the capability to set ClassAd attributes for a condor_dagman job within the DAG file by using the new SET_JOB_ATTR command. (See section 2.10.9 for details.) (Ticket #5107).

• The dagman.out file produced by condor_dagman now has event timestamps added to the lines that report condor_dagman reading a log event. For example:

    01/13/16 11:29:03 Event: ULOG_SUBMIT for HTCondor Node NodeA (674.0.0) {01/13/16 11:28:59}
  

  The timestamp in curly brackets at the end is the actual timestamp of the event. (Ticket #5439).

• The -run and -hold arguments of condor_q used to produce garbage output when used with other formatting options such as -format. Now they will always constrain the query, and will also set the output format when used by themselves. (Ticket #11).

• The condor_status -verbose argument has been removed; the equivalent -long argument should be used instead.

Bugs Fixed:

• On Windows, configuring HTCondor to restrict the range of outbound port numbers may cause substantial delays when using the command-line tools. Since we now know that it's not free to do so, LOWPORT and HIGHPORT no longer restrict the port numbers of outbound connections on Windows. If you still require this functionality, use OUT_LOWPORT and OUT_HIGHPORT. (Ticket #4711).

• Fixed a bug that could cause a daemon to be in the wrong privilege state when attempting to act as the user. (Ticket #5467).

Version 8.5.1

Release Notes:

• HTCondor version 8.5.1 released on December 21, 2015.

• Shared port is enabled by default. (Ticket #3813). (Ticket #5103).

New Features:

• The condor_startd history file now contains the peak memory usage, by an exited job, not the more recent. (Ticket #5436).

• When the condor_starter evicts a job, perhaps because it has exceeded a memory limit, it does not transfer back to the submit machine the sandbox of working files. This is consistent with other types of holds. (Ticket #5437).

• The condor_startd now advertises the following attributes on Linux machines: CpuFamily CpuModelNumber CacheSize. These are pulled from the /proc/cpuinfo file. (Ticket #5323).

• condor_q has a new option -schedd-constraint which can be used to constrain the queues displayed when using the -global option. (Ticket #5043).

• When an HTCondor-C job is submitted to a remote condor_schedd, the remote job ad now includes the attribute SubmitterGlobalJobId, whose value is the same as the attribute GlobalJobId in the original HTCondor-C job. (Ticket #3472).

• The condor_schedd now sets environment variables for scheduler universe jobs so that the jobs can more easily find the condor_schedd's contact information. On machines where there are multiple condor_schedds running, this helps DAGMan and similar applications contact the condor_schedd that started them. (Ticket #5166).

• When SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION is set to True, the related authorizations are now automatically enabled. Previously, submit-side@matchsession and execute-side@matchsession entries had to be added to the ALLOW_DAEMON and ALLOW_CLIENT (if set) authorization parameters in order for this feature to work. (Ticket #5304).

Bugs Fixed:

• condor_history run on a pool with partitionable slots now shows the correct dynamic slot. (Ticket #4261).

Version 8.5.0

Release Notes:

• HTCondor version 8.5.0 released on October 12, 2015.

New Features:

• The condor_startd history file contains two new attributes: BadputCausedByDraining and BadputCausedByPreemption, two boolean-valued attributes which are true if the job was evicted not by a user request. (Ticket #5255).

• The python bindings have a new Claim API, allowing Computing-On-Demand (COD) to be invoked via python. (Ticket #5130).

• The python bindings can now submit multiple distinct processes using the submitMany method, similar to a condor_submit file with multiple queue statements. (Ticket #4916).

• The python bindings now provide improved support for managing multiple concurrent queries. (Ticket #5187).

• As an experimental feature, the python bindings implement the HTCondor negotiation protocol. (Ticket #5125).

• Changed "Condor" to "HTCondor" in condor_dagman output (mainly in the dagman.out file). (Ticket #5144).

• The new configuration parameter JOB_SPOOL_PERMISSIONS controls the permissions on a job's spool directory managed by the condor_schedd on unix. It defaults to the value user, which results in a permissions value of 0700. Other valid values are group (permissions 0750) and world (permissions 0755). Previously, all job spool directories had access permissions of 0755. (Ticket #4896).

• The condor_schedd no longer changes the ownership of spooled job files that it manages. Now, the files are always owned by the submitting user. The previous behavior of changing ownership to/from the condor account can be restored by setting the new configuration parameter CHOWN_JOB_SPOOL_FILES to True. (Ticket #5226).

Bugs Fixed:

• None.