Job scheduler


Job scheduler

A job scheduler is a software application that is in charge of unattended background executions, commonly known for historical reasons as batch processing.[1]

Synonyms are batch system, Distributed Resource Management System (DRMS), and Distributed Resource Manager (DRM). Today's job schedulers typically provide a graphical user interface and a single point of control for definition and monitoring of background executions in a distributed network of computers. Increasingly job schedulers are required to orchestrate the integration of real-time business activities with traditional background IT processing, across different operating system platforms and business application environments.

Job scheduling should not be confused with process scheduling, which is the assignment of currently running processes to CPUs by the operating system.

Contents

Overview

Basic features expected of job scheduler software include:

  • interfaces which help to define workflows and/or job dependencies
  • automatic submission of executions
  • interfaces to monitor the executions
  • priorities and/or queues to control the execution order of unrelated jobs

If software from a completely different area includes all or some of those features, this software is considered[by whom?] to have job scheduling capabilities.

Most operating systems (such as Unix and Windows) provide basic job scheduling capabilities, for example: cron. Web hosting services provide job scheduling capabilities through a control panel or a webcron solution. Many programs such as DBMS, backup, ERPs, and BPM also include relevant job-scheduling capabilities. Operating-system (OS) or point program supplied job-scheduling will not usually provide the ability to schedule beyond a single OS instance or outside the remit of the specific program. Organizations needing to automate highly complex related and un-related IT workload may also leverage more-advanced features from a job scheduler, such as:

  • real-time scheduling based on external, un-predictable events
  • automatic restart and recovery in event of failures
  • alerting and notification to operations personnel
  • generation of incident reports
  • audit trails for regulatory compliance purposes

These advanced capabilities can be written by in-house developers but are more often provided by suppliers who specialize in systems-management software.

Main concepts

There are many concepts that are central to almost every job scheduler implementation and that are widely recognized with minimal variations:

  • Jobs
  • Dependencies
  • Job Streams
  • Users

Beyond the basic, single OS instance scheduling tools there are two major architectures that exist for Job Scheduling software.

  • Master/Agent architecture — the historic architecture for Job scheduling software. The Job Scheduling software is installed on a single machine (Master) while on production machines only a very small component (Agent) is installed that awaits commands from the Master, executes them, and returns the exit code back to the Master.
  • Cooperative architecture — a decentralized model where each machine is capable of helping with scheduling and can offload locally scheduled jobs to other cooperating machines. This enables dynamic workload balancing to maximize hardware resource utilization and high availability to ensure service delivery.

Batch queuing for HPC clusters

An important niche for job schedulers is managing the job queue for a cluster of computers. Typically, the scheduler will schedule jobs from the queue as sufficient resources (cluster nodes) become idle. Some widely used cluster batch systems are Oracle Grid Engine, Portable Batch System, LoadLeveler, Condor, OAR and Simple Linux Utility for Resource Management.

History

Job Scheduling has a long history. Job Schedulers are one of the major components of the IT infrastructure since the early mainframe systems. At first, stacks of punched cards were processed one after the other, hence the term “batch processing.”

From a historical point of view, we can distinguish two main eras about Job Schedulers:

  1. The mainframe era
    • Job Control Language (JCL) on IBM mainframes. Initially based on JCL functionality to handle dependencies this era is typified by the development of sophisticated scheduling solutions forming part of the systems management and automation toolset on the mainframe.
  2. The open systems era
    • Modern schedulers on a variety of architectures and operating systems. With standard scheduling tools limited to such as Cron, the need for mainframe standard job schedulers has grown with the increased adoption of distributed computing environments.

In terms of the type of scheduling there are also distinct eras:

  1. Batch processing - the traditional date and time based execution of background tasks based on a defined period during which resources were available for batch processing (the batch window). In effect the original mainframe approach transposed onto the open systems environment.
  2. Event-driven process automation - where background processes cannot be simply run at a defined time, either because the nature of the business demands that workload is based on the occurrence of external events (such as the arrival of an order from a customer or a stock update from a store branch) or because there is no / insufficient batch window.
  3. Service Oriented job scheduling - recent developments in Service Oriented Architecture (SOA) have seen a move towards deploying job scheduling as a reusable IT infrastructure service that can play a role in the integration of existing business application workload with new Web Services based real-time applications.

Scheduling

Various schemes are used to decide which particular job to run. Parameters that might be considered include:

  • Job priority
  • Compute resource availability
  • License key if job is using licensed software
  • Execution time allocated to user
  • Number of simultaneous jobs allowed for a user
  • Estimated execution time
  • Elapsed execution time
  • Availability of peripheral devices
  • Occurrence of prescribed events

Common batch systems

See also

Further reading

References

  1. ^ Effect of Job Size Characteristics on Job Scheduling Performance

Wikimedia Foundation. 2010.

Look at other dictionaries:

  • Open Source Job Scheduler — Dieser Artikel wurde aufgrund von inhaltlichen Mängeln auf der Qualitätssicherungsseite der Redaktion Informatik eingetragen. Dies geschieht, um die Qualität der Artikel aus dem Themengebiet Informatik auf ein akzeptables Niveau zu bringen. Hilf… …   Deutsch Wikipedia

  • Open Source Job Scheduler — Developer(s) Software und Organisations Service GmbH Stable release 1.3.7 / April 23, 2010; 18 months ago (2010 04 23) …   Wikipedia

  • Job (software) — In computing a job (or process) is a term used to refer to a single instance of a program. The term is mostly used on multitasking systems. These are operating systems that are able to process many tasks concurrently ; however, this is normally… …   Wikipedia

  • Job control — in computing refers to the control of multiple tasks or Jobs on a computer system, ensuring that they each have access to adequate resources to perform correctly, that competition for limited resources does not cause a deadlock where two or more… …   Wikipedia

  • Job Submission Description Language — is an extensible XML specification from the Global Grid Forum for the description of simple tasks to non interactive computer execution systems. Currently at version 1.0 (released November 7, 2005), the specification focuses on the description of …   Wikipedia

  • scheduler — UK US /ˈʃedjuːlər/ US  /ˈskedʒuːlər/ noun [C] PRODUCTION, COMMERCE, TRANSPORT ► someone whose job is to create or work with schedules …   Financial and business terms

  • Scheduler (Informatik) — Ein Prozess Scheduler (Scheduler = Steuerprogramm) ist eine Arbitrationslogik, der die zeitliche Ausführung mehrerer Prozesse in Betriebssystemen regelt. Prozess Scheduler kann man grob in unterbrechende (preemptive) und nicht unterbrechende (non …   Deutsch Wikipedia

  • JOB — General Employment Enterprises (Business » AMEX Symbols) Just Over Broke (Internet » Chat) * Journey Of The Broke (Business » Positions) * Job Opportunity Bank (Business » Positions) * Japanese Ocean Bomb (Governmental » Military) * Just… …   Abbreviations dictionary

  • Maui Cluster Scheduler — is a job scheduler for use on clusters and supercomputers initially developed by Cluster Resources, Inc.. Maui is capable of supporting multiple scheduling policies, dynamic priorities, reservations, and fairshare capabilities. Maui satisfies… …   Wikipedia

  • IBM Tivoli Workload Scheduler LoadLeveler — Infobox Software name = Tivoli Workload Scheduler LoadLeveler caption = developer = IBM latest release version = 3.4 latest release date = October 2006 operating system = AIX / Linux genre = job scheduler license = Proprietary website =… …   Wikipedia