Docs Packages
  1. Packages
  2. Google Cloud Native
  3. API Docs
  4. dataproc
  5. dataproc/v1
  6. getJob

Google Cloud Native is in preview. Google Cloud Classic is fully supported.

Google Cloud Native v0.32.0 published on Wednesday, Nov 29, 2023 by Pulumi
pulumi/pulumi-google-native

google-native.dataproc/v1.getJob

Explore with Pulumi AI

google-native logo

Google Cloud Native is in preview. Google Cloud Classic is fully supported.

Google Cloud Native v0.32.0 published on Wednesday, Nov 29, 2023 by Pulumi
pulumi/pulumi-google-native

    Gets the resource representation for a job in a project.

    Using getJob

    Two invocation forms are available. The direct form accepts plain arguments and either blocks until the result value is available, or returns a Promise-wrapped result. The output form accepts Input-wrapped arguments and returns an Output-wrapped result.

    function getJob(args: GetJobArgs, opts?: InvokeOptions): Promise<GetJobResult>
function getJobOutput(args: GetJobOutputArgs, opts?: InvokeOptions): Output<GetJobResult>
    def get_job(job_id: Optional[str] = None,
            project: Optional[str] = None,
            region: Optional[str] = None,
            opts: Optional[InvokeOptions] = None) -> GetJobResult
def get_job_output(job_id: Optional[pulumi.Input[str]] = None,
            project: Optional[pulumi.Input[str]] = None,
            region: Optional[pulumi.Input[str]] = None,
            opts: Optional[InvokeOptions] = None) -> Output[GetJobResult]
    func LookupJob(ctx *Context, args *LookupJobArgs, opts ...InvokeOption) (*LookupJobResult, error)
func LookupJobOutput(ctx *Context, args *LookupJobOutputArgs, opts ...InvokeOption) LookupJobResultOutput

    > Note: This function is named LookupJob in the Go SDK.

    public static class GetJob 
{
    public static Task<GetJobResult> InvokeAsync(GetJobArgs args, InvokeOptions? opts = null)
    public static Output<GetJobResult> Invoke(GetJobInvokeArgs args, InvokeOptions? opts = null)
}
    public static CompletableFuture<GetJobResult> getJob(GetJobArgs args, InvokeOptions options)
// Output-based functions aren't available in Java yet

    fn::invoke:
  function: google-native:dataproc/v1:getJob
  arguments:
    # arguments dictionary

    The following arguments are supported:

    JobId string
    Region string
    Project string
    JobId string
    Region string
    Project string
    jobId String
    region String
    project String
    jobId string
    region string
    project string
    jobId String
    region String
    project String

    getJob Result

    The following output properties are available:

    Done bool
    Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled.
    DriverControlFilesUri string
    If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri.
    DriverOutputResourceUri string
    A URI pointing to the location of the stdout of the job's driver program.
    DriverSchedulingConfig Pulumi.GoogleNative.Dataproc.V1.Outputs.DriverSchedulingConfigResponse
    Optional. Driver scheduling configuration.
    FlinkJob Pulumi.GoogleNative.Dataproc.V1.Outputs.FlinkJobResponse
    Optional. Job is a Flink job.
    HadoopJob Pulumi.GoogleNative.Dataproc.V1.Outputs.HadoopJobResponse
    Optional. Job is a Hadoop job.
    HiveJob Pulumi.GoogleNative.Dataproc.V1.Outputs.HiveJobResponse
    Optional. Job is a Hive job.
    JobUuid string
    A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time.
    Labels Dictionary<string, string>
    Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
    PigJob Pulumi.GoogleNative.Dataproc.V1.Outputs.PigJobResponse
    Optional. Job is a Pig job.
    Placement Pulumi.GoogleNative.Dataproc.V1.Outputs.JobPlacementResponse
    Job information, including how, when, and where to run the job.
    PrestoJob Pulumi.GoogleNative.Dataproc.V1.Outputs.PrestoJobResponse
    Optional. Job is a Presto job.
    PysparkJob Pulumi.GoogleNative.Dataproc.V1.Outputs.PySparkJobResponse
    Optional. Job is a PySpark job.
    Reference Pulumi.GoogleNative.Dataproc.V1.Outputs.JobReferenceResponse
    Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a job_id.
    Scheduling Pulumi.GoogleNative.Dataproc.V1.Outputs.JobSchedulingResponse
    Optional. Job scheduling configuration.
    SparkJob Pulumi.GoogleNative.Dataproc.V1.Outputs.SparkJobResponse
    Optional. Job is a Spark job.
    SparkRJob Pulumi.GoogleNative.Dataproc.V1.Outputs.SparkRJobResponse
    Optional. Job is a SparkR job.
    SparkSqlJob Pulumi.GoogleNative.Dataproc.V1.Outputs.SparkSqlJobResponse
    Optional. Job is a SparkSql job.
    Status Pulumi.GoogleNative.Dataproc.V1.Outputs.JobStatusResponse
    The job status. Additional application-specific status information might be contained in the type_job and yarn_applications fields.
    StatusHistory List<Pulumi.GoogleNative.Dataproc.V1.Outputs.JobStatusResponse>
    The previous job status.
    TrinoJob Pulumi.GoogleNative.Dataproc.V1.Outputs.TrinoJobResponse
    Optional. Job is a Trino job.
    YarnApplications List<Pulumi.GoogleNative.Dataproc.V1.Outputs.YarnApplicationResponse>
    The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release.
    Done bool
    Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled.
    DriverControlFilesUri string
    If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri.
    DriverOutputResourceUri string
    A URI pointing to the location of the stdout of the job's driver program.
    DriverSchedulingConfig DriverSchedulingConfigResponse
    Optional. Driver scheduling configuration.
    FlinkJob FlinkJobResponse
    Optional. Job is a Flink job.
    HadoopJob HadoopJobResponse
    Optional. Job is a Hadoop job.
    HiveJob HiveJobResponse
    Optional. Job is a Hive job.
    JobUuid string
    A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time.
    Labels map[string]string
    Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
    PigJob PigJobResponse
    Optional. Job is a Pig job.
    Placement JobPlacementResponse
    Job information, including how, when, and where to run the job.
    PrestoJob PrestoJobResponse
    Optional. Job is a Presto job.
    PysparkJob PySparkJobResponse
    Optional. Job is a PySpark job.
    Reference JobReferenceResponse
    Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a job_id.
    Scheduling JobSchedulingResponse
    Optional. Job scheduling configuration.
    SparkJob SparkJobResponse
    Optional. Job is a Spark job.
    SparkRJob SparkRJobResponse
    Optional. Job is a SparkR job.
    SparkSqlJob SparkSqlJobResponse
    Optional. Job is a SparkSql job.
    Status JobStatusResponse
    The job status. Additional application-specific status information might be contained in the type_job and yarn_applications fields.
    StatusHistory []JobStatusResponse
    The previous job status.
    TrinoJob TrinoJobResponse
    Optional. Job is a Trino job.
    YarnApplications []YarnApplicationResponse
    The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release.
    done Boolean
    Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled.
    driverControlFilesUri String
    If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri.
    driverOutputResourceUri String
    A URI pointing to the location of the stdout of the job's driver program.
    driverSchedulingConfig DriverSchedulingConfigResponse
    Optional. Driver scheduling configuration.
    flinkJob FlinkJobResponse
    Optional. Job is a Flink job.
    hadoopJob HadoopJobResponse
    Optional. Job is a Hadoop job.
    hiveJob HiveJobResponse
    Optional. Job is a Hive job.
    jobUuid String
    A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time.
    labels Map<String,String>
    Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
    pigJob PigJobResponse
    Optional. Job is a Pig job.
    placement JobPlacementResponse
    Job information, including how, when, and where to run the job.
    prestoJob PrestoJobResponse
    Optional. Job is a Presto job.
    pysparkJob PySparkJobResponse
    Optional. Job is a PySpark job.
    reference JobReferenceResponse
    Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a job_id.
    scheduling JobSchedulingResponse
    Optional. Job scheduling configuration.
    sparkJob SparkJobResponse
    Optional. Job is a Spark job.
    sparkRJob SparkRJobResponse
    Optional. Job is a SparkR job.
    sparkSqlJob SparkSqlJobResponse
    Optional. Job is a SparkSql job.
    status JobStatusResponse
    The job status. Additional application-specific status information might be contained in the type_job and yarn_applications fields.
    statusHistory List<JobStatusResponse>
    The previous job status.
    trinoJob TrinoJobResponse
    Optional. Job is a Trino job.
    yarnApplications List<YarnApplicationResponse>
    The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release.
    done boolean
    Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled.
    driverControlFilesUri string
    If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri.
    driverOutputResourceUri string
    A URI pointing to the location of the stdout of the job's driver program.
    driverSchedulingConfig DriverSchedulingConfigResponse
    Optional. Driver scheduling configuration.
    flinkJob FlinkJobResponse
    Optional. Job is a Flink job.
    hadoopJob HadoopJobResponse
    Optional. Job is a Hadoop job.
    hiveJob HiveJobResponse
    Optional. Job is a Hive job.
    jobUuid string
    A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time.
    labels {[key: string]: string}
    Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
    pigJob PigJobResponse
    Optional. Job is a Pig job.
    placement JobPlacementResponse
    Job information, including how, when, and where to run the job.
    prestoJob PrestoJobResponse
    Optional. Job is a Presto job.
    pysparkJob PySparkJobResponse
    Optional. Job is a PySpark job.
    reference JobReferenceResponse
    Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a job_id.
    scheduling JobSchedulingResponse
    Optional. Job scheduling configuration.
    sparkJob SparkJobResponse
    Optional. Job is a Spark job.
    sparkRJob SparkRJobResponse
    Optional. Job is a SparkR job.
    sparkSqlJob SparkSqlJobResponse
    Optional. Job is a SparkSql job.
    status JobStatusResponse
    The job status. Additional application-specific status information might be contained in the type_job and yarn_applications fields.
    statusHistory JobStatusResponse[]
    The previous job status.
    trinoJob TrinoJobResponse
    Optional. Job is a Trino job.
    yarnApplications YarnApplicationResponse[]
    The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release.
    done bool
    Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled.
    driver_control_files_uri str
    If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri.
    driver_output_resource_uri str
    A URI pointing to the location of the stdout of the job's driver program.
    driver_scheduling_config DriverSchedulingConfigResponse
    Optional. Driver scheduling configuration.
    flink_job FlinkJobResponse
    Optional. Job is a Flink job.
    hadoop_job HadoopJobResponse
    Optional. Job is a Hadoop job.
    hive_job HiveJobResponse
    Optional. Job is a Hive job.
    job_uuid str
    A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time.
    labels Mapping[str, str]
    Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
    pig_job PigJobResponse
    Optional. Job is a Pig job.
    placement JobPlacementResponse
    Job information, including how, when, and where to run the job.
    presto_job PrestoJobResponse
    Optional. Job is a Presto job.
    pyspark_job PySparkJobResponse
    Optional. Job is a PySpark job.
    reference JobReferenceResponse
    Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a job_id.
    scheduling JobSchedulingResponse
    Optional. Job scheduling configuration.
    spark_job SparkJobResponse
    Optional. Job is a Spark job.
    spark_r_job SparkRJobResponse
    Optional. Job is a SparkR job.
    spark_sql_job SparkSqlJobResponse
    Optional. Job is a SparkSql job.
    status JobStatusResponse
    The job status. Additional application-specific status information might be contained in the type_job and yarn_applications fields.
    status_history Sequence[JobStatusResponse]
    The previous job status.
    trino_job TrinoJobResponse
    Optional. Job is a Trino job.
    yarn_applications Sequence[YarnApplicationResponse]
    The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release.
    done Boolean
    Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled.
    driverControlFilesUri String
    If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri.
    driverOutputResourceUri String
    A URI pointing to the location of the stdout of the job's driver program.
    driverSchedulingConfig Property Map
    Optional. Driver scheduling configuration.
    flinkJob Property Map
    Optional. Job is a Flink job.
    hadoopJob Property Map
    Optional. Job is a Hadoop job.
    hiveJob Property Map
    Optional. Job is a Hive job.
    jobUuid String
    A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time.
    labels Map<String>
    Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a job.
    pigJob Property Map
    Optional. Job is a Pig job.
    placement Property Map
    Job information, including how, when, and where to run the job.
    prestoJob Property Map
    Optional. Job is a Presto job.
    pysparkJob Property Map
    Optional. Job is a PySpark job.
    reference Property Map
    Optional. The fully qualified reference to the job, which can be used to obtain the equivalent REST path of the job resource. If this property is not specified when a job is created, the server generates a job_id.
    scheduling Property Map
    Optional. Job scheduling configuration.
    sparkJob Property Map
    Optional. Job is a Spark job.
    sparkRJob Property Map
    Optional. Job is a SparkR job.
    sparkSqlJob Property Map
    Optional. Job is a SparkSql job.
    status Property Map
    The job status. Additional application-specific status information might be contained in the type_job and yarn_applications fields.
    statusHistory List<Property Map>
    The previous job status.
    trinoJob Property Map
    Optional. Job is a Trino job.
    yarnApplications List<Property Map>
    The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release.

    Supporting Types

    DriverSchedulingConfigResponse

    MemoryMb int
    The amount of memory in MB the driver is requesting.
    Vcores int
    The number of vCPUs the driver is requesting.
    MemoryMb int
    The amount of memory in MB the driver is requesting.
    Vcores int
    The number of vCPUs the driver is requesting.
    memoryMb Integer
    The amount of memory in MB the driver is requesting.
    vcores Integer
    The number of vCPUs the driver is requesting.
    memoryMb number
    The amount of memory in MB the driver is requesting.
    vcores number
    The number of vCPUs the driver is requesting.
    memory_mb int
    The amount of memory in MB the driver is requesting.
    vcores int
    The number of vCPUs the driver is requesting.
    memoryMb Number
    The amount of memory in MB the driver is requesting.
    vcores Number
    The number of vCPUs the driver is requesting.

    FlinkJobResponse

    Args List<string>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    JarFileUris List<string>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Flink driver and tasks.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainClass string
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jarFileUris.
    MainJarFileUri string
    The HCFS URI of the jar file that contains the main class.
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure Flink. Properties that conflict with values set by the Dataproc API might beoverwritten. Can include properties set in/etc/flink/conf/flink-defaults.conf and classes in user code.
    SavepointUri string
    Optional. HCFS URI of the savepoint, which contains the last saved progress for starting the current job.
    Args []string
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    JarFileUris []string
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Flink driver and tasks.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainClass string
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jarFileUris.
    MainJarFileUri string
    The HCFS URI of the jar file that contains the main class.
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure Flink. Properties that conflict with values set by the Dataproc API might beoverwritten. Can include properties set in/etc/flink/conf/flink-defaults.conf and classes in user code.
    SavepointUri string
    Optional. HCFS URI of the savepoint, which contains the last saved progress for starting the current job.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Flink driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainClass String
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jarFileUris.
    mainJarFileUri String
    The HCFS URI of the jar file that contains the main class.
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure Flink. Properties that conflict with values set by the Dataproc API might beoverwritten. Can include properties set in/etc/flink/conf/flink-defaults.conf and classes in user code.
    savepointUri String
    Optional. HCFS URI of the savepoint, which contains the last saved progress for starting the current job.
    args string[]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    jarFileUris string[]
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Flink driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainClass string
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jarFileUris.
    mainJarFileUri string
    The HCFS URI of the jar file that contains the main class.
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure Flink. Properties that conflict with values set by the Dataproc API might beoverwritten. Can include properties set in/etc/flink/conf/flink-defaults.conf and classes in user code.
    savepointUri string
    Optional. HCFS URI of the savepoint, which contains the last saved progress for starting the current job.
    args Sequence[str]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    jar_file_uris Sequence[str]
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Flink driver and tasks.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    main_class str
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jarFileUris.
    main_jar_file_uri str
    The HCFS URI of the jar file that contains the main class.
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure Flink. Properties that conflict with values set by the Dataproc API might beoverwritten. Can include properties set in/etc/flink/conf/flink-defaults.conf and classes in user code.
    savepoint_uri str
    Optional. HCFS URI of the savepoint, which contains the last saved progress for starting the current job.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Flink driver and tasks.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    mainClass String
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jarFileUris.
    mainJarFileUri String
    The HCFS URI of the jar file that contains the main class.
    properties Map<String>
    Optional. A mapping of property names to values, used to configure Flink. Properties that conflict with values set by the Dataproc API might beoverwritten. Can include properties set in/etc/flink/conf/flink-defaults.conf and classes in user code.
    savepointUri String
    Optional. HCFS URI of the savepoint, which contains the last saved progress for starting the current job.

    HadoopJobResponse

    ArchiveUris List<string>
    Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
    Args List<string>
    Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    FileUris List<string>
    Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
    JarFileUris List<string>
    Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainClass string
    The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
    MainJarFileUri string
    The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
    ArchiveUris []string
    Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
    Args []string
    Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    FileUris []string
    Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
    JarFileUris []string
    Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainClass string
    The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
    MainJarFileUri string
    The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
    jarFileUris List<String>
    Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainClass String
    The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
    mainJarFileUri String
    The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
    archiveUris string[]
    Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
    args string[]
    Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    fileUris string[]
    Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
    jarFileUris string[]
    Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainClass string
    The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
    mainJarFileUri string
    The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
    archive_uris Sequence[str]
    Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
    args Sequence[str]
    Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    file_uris Sequence[str]
    Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
    jar_file_uris Sequence[str]
    Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    main_class str
    The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
    main_jar_file_uri str
    The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision might occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.
    jarFileUris List<String>
    Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    mainClass String
    The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.
    mainJarFileUri String
    The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'
    properties Map<String>
    Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.

    HiveJobResponse

    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    JarFileUris List<string>
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
    Properties Dictionary<string, string>
    Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
    QueryFileUri string
    The HCFS URI of the script that contains Hive queries.
    QueryList Pulumi.GoogleNative.Dataproc.V1.Inputs.QueryListResponse
    A list of queries.
    ScriptVariables Dictionary<string, string>
    Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    JarFileUris []string
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
    Properties map[string]string
    Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
    QueryFileUri string
    The HCFS URI of the script that contains Hive queries.
    QueryList QueryListResponse
    A list of queries.
    ScriptVariables map[string]string
    Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
    properties Map<String,String>
    Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
    queryFileUri String
    The HCFS URI of the script that contains Hive queries.
    queryList QueryListResponse
    A list of queries.
    scriptVariables Map<String,String>
    Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
    continueOnFailure boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jarFileUris string[]
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
    properties {[key: string]: string}
    Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
    queryFileUri string
    The HCFS URI of the script that contains Hive queries.
    queryList QueryListResponse
    A list of queries.
    scriptVariables {[key: string]: string}
    Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
    continue_on_failure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jar_file_uris Sequence[str]
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
    properties Mapping[str, str]
    Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
    query_file_uri str
    The HCFS URI of the script that contains Hive queries.
    query_list QueryListResponse
    A list of queries.
    script_variables Mapping[str, str]
    Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Hive server and Hadoop MapReduce (MR) tasks. Can contain Hive SerDes and UDFs.
    properties Map<String>
    Optional. A mapping of property names and values, used to configure Hive. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/hive/conf/hive-site.xml, and classes in user code.
    queryFileUri String
    The HCFS URI of the script that contains Hive queries.
    queryList Property Map
    A list of queries.
    scriptVariables Map<String>
    Optional. Mapping of query variable names to values (equivalent to the Hive command: SET name="value";).

    JobPlacementResponse

    ClusterLabels Dictionary<string, string>
    Optional. Cluster labels to identify a cluster where the job will be submitted.
    ClusterName string
    The name of the cluster where the job will be submitted.
    ClusterUuid string
    A cluster UUID generated by the Dataproc service when the job is submitted.
    ClusterLabels map[string]string
    Optional. Cluster labels to identify a cluster where the job will be submitted.
    ClusterName string
    The name of the cluster where the job will be submitted.
    ClusterUuid string
    A cluster UUID generated by the Dataproc service when the job is submitted.
    clusterLabels Map<String,String>
    Optional. Cluster labels to identify a cluster where the job will be submitted.
    clusterName String
    The name of the cluster where the job will be submitted.
    clusterUuid String
    A cluster UUID generated by the Dataproc service when the job is submitted.
    clusterLabels {[key: string]: string}
    Optional. Cluster labels to identify a cluster where the job will be submitted.
    clusterName string
    The name of the cluster where the job will be submitted.
    clusterUuid string
    A cluster UUID generated by the Dataproc service when the job is submitted.
    cluster_labels Mapping[str, str]
    Optional. Cluster labels to identify a cluster where the job will be submitted.
    cluster_name str
    The name of the cluster where the job will be submitted.
    cluster_uuid str
    A cluster UUID generated by the Dataproc service when the job is submitted.
    clusterLabels Map<String>
    Optional. Cluster labels to identify a cluster where the job will be submitted.
    clusterName String
    The name of the cluster where the job will be submitted.
    clusterUuid String
    A cluster UUID generated by the Dataproc service when the job is submitted.

    JobReferenceResponse

    JobId string
    Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
    Project string
    Optional. The ID of the Google Cloud Platform project that the job belongs to. If specified, must match the request project ID.
    JobId string
    Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
    Project string
    Optional. The ID of the Google Cloud Platform project that the job belongs to. If specified, must match the request project ID.
    jobId String
    Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
    project String
    Optional. The ID of the Google Cloud Platform project that the job belongs to. If specified, must match the request project ID.
    jobId string
    Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
    project string
    Optional. The ID of the Google Cloud Platform project that the job belongs to. If specified, must match the request project ID.
    job_id str
    Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
    project str
    Optional. The ID of the Google Cloud Platform project that the job belongs to. If specified, must match the request project ID.
    jobId String
    Optional. The job ID, which must be unique within the project.The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or hyphens (-). The maximum length is 100 characters.If not specified by the caller, the job ID will be provided by the server.
    project String
    Optional. The ID of the Google Cloud Platform project that the job belongs to. If specified, must match the request project ID.

    JobSchedulingResponse

    MaxFailuresPerHour int
    Optional. Maximum number of times per hour a driver can be restarted as a result of driver exiting with non-zero code before job is reported failed.A job might be reported as thrashing if the driver exits with a non-zero code four times within a 10-minute window.Maximum value is 10.Note: This restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    MaxFailuresTotal int
    Optional. Maximum total number of times a driver can be restarted as a result of the driver exiting with a non-zero code. After the maximum number is reached, the job will be reported as failed.Maximum value is 240.Note: Currently, this restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    MaxFailuresPerHour int
    Optional. Maximum number of times per hour a driver can be restarted as a result of driver exiting with non-zero code before job is reported failed.A job might be reported as thrashing if the driver exits with a non-zero code four times within a 10-minute window.Maximum value is 10.Note: This restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    MaxFailuresTotal int
    Optional. Maximum total number of times a driver can be restarted as a result of the driver exiting with a non-zero code. After the maximum number is reached, the job will be reported as failed.Maximum value is 240.Note: Currently, this restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    maxFailuresPerHour Integer
    Optional. Maximum number of times per hour a driver can be restarted as a result of driver exiting with non-zero code before job is reported failed.A job might be reported as thrashing if the driver exits with a non-zero code four times within a 10-minute window.Maximum value is 10.Note: This restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    maxFailuresTotal Integer
    Optional. Maximum total number of times a driver can be restarted as a result of the driver exiting with a non-zero code. After the maximum number is reached, the job will be reported as failed.Maximum value is 240.Note: Currently, this restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    maxFailuresPerHour number
    Optional. Maximum number of times per hour a driver can be restarted as a result of driver exiting with non-zero code before job is reported failed.A job might be reported as thrashing if the driver exits with a non-zero code four times within a 10-minute window.Maximum value is 10.Note: This restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    maxFailuresTotal number
    Optional. Maximum total number of times a driver can be restarted as a result of the driver exiting with a non-zero code. After the maximum number is reached, the job will be reported as failed.Maximum value is 240.Note: Currently, this restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    max_failures_per_hour int
    Optional. Maximum number of times per hour a driver can be restarted as a result of driver exiting with non-zero code before job is reported failed.A job might be reported as thrashing if the driver exits with a non-zero code four times within a 10-minute window.Maximum value is 10.Note: This restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    max_failures_total int
    Optional. Maximum total number of times a driver can be restarted as a result of the driver exiting with a non-zero code. After the maximum number is reached, the job will be reported as failed.Maximum value is 240.Note: Currently, this restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    maxFailuresPerHour Number
    Optional. Maximum number of times per hour a driver can be restarted as a result of driver exiting with non-zero code before job is reported failed.A job might be reported as thrashing if the driver exits with a non-zero code four times within a 10-minute window.Maximum value is 10.Note: This restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).
    maxFailuresTotal Number
    Optional. Maximum total number of times a driver can be restarted as a result of the driver exiting with a non-zero code. After the maximum number is reached, the job will be reported as failed.Maximum value is 240.Note: Currently, this restartable job option is not supported in Dataproc workflow templates (https://cloud.google.com/dataproc/docs/concepts/workflows/using-workflows#adding_jobs_to_a_template).

    JobStatusResponse

    Details string
    Optional. Output only. Job state details, such as an error description if the state is ERROR.
    State string
    A state message specifying the overall job state.
    StateStartTime string
    The time when this state was entered.
    Substate string
    Additional state information, which includes status reported by the agent.
    Details string
    Optional. Output only. Job state details, such as an error description if the state is ERROR.
    State string
    A state message specifying the overall job state.
    StateStartTime string
    The time when this state was entered.
    Substate string
    Additional state information, which includes status reported by the agent.
    details String
    Optional. Output only. Job state details, such as an error description if the state is ERROR.
    state String
    A state message specifying the overall job state.
    stateStartTime String
    The time when this state was entered.
    substate String
    Additional state information, which includes status reported by the agent.
    details string
    Optional. Output only. Job state details, such as an error description if the state is ERROR.
    state string
    A state message specifying the overall job state.
    stateStartTime string
    The time when this state was entered.
    substate string
    Additional state information, which includes status reported by the agent.
    details str
    Optional. Output only. Job state details, such as an error description if the state is ERROR.
    state str
    A state message specifying the overall job state.
    state_start_time str
    The time when this state was entered.
    substate str
    Additional state information, which includes status reported by the agent.
    details String
    Optional. Output only. Job state details, such as an error description if the state is ERROR.
    state String
    A state message specifying the overall job state.
    stateStartTime String
    The time when this state was entered.
    substate String
    Additional state information, which includes status reported by the agent.

    LoggingConfigResponse

    DriverLogLevels Dictionary<string, string>
    The per-package log levels for the driver. This can include "root" package name to configure rootLogger. Examples: - 'com.google = FATAL' - 'root = INFO' - 'org.apache = DEBUG'
    DriverLogLevels map[string]string
    The per-package log levels for the driver. This can include "root" package name to configure rootLogger. Examples: - 'com.google = FATAL' - 'root = INFO' - 'org.apache = DEBUG'
    driverLogLevels Map<String,String>
    The per-package log levels for the driver. This can include "root" package name to configure rootLogger. Examples: - 'com.google = FATAL' - 'root = INFO' - 'org.apache = DEBUG'
    driverLogLevels {[key: string]: string}
    The per-package log levels for the driver. This can include "root" package name to configure rootLogger. Examples: - 'com.google = FATAL' - 'root = INFO' - 'org.apache = DEBUG'
    driver_log_levels Mapping[str, str]
    The per-package log levels for the driver. This can include "root" package name to configure rootLogger. Examples: - 'com.google = FATAL' - 'root = INFO' - 'org.apache = DEBUG'
    driverLogLevels Map<String>
    The per-package log levels for the driver. This can include "root" package name to configure rootLogger. Examples: - 'com.google = FATAL' - 'root = INFO' - 'org.apache = DEBUG'

    PigJobResponse

    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    JarFileUris List<string>
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
    QueryFileUri string
    The HCFS URI of the script that contains the Pig queries.
    QueryList Pulumi.GoogleNative.Dataproc.V1.Inputs.QueryListResponse
    A list of queries.
    ScriptVariables Dictionary<string, string>
    Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    JarFileUris []string
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
    QueryFileUri string
    The HCFS URI of the script that contains the Pig queries.
    QueryList QueryListResponse
    A list of queries.
    ScriptVariables map[string]string
    Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
    queryFileUri String
    The HCFS URI of the script that contains the Pig queries.
    queryList QueryListResponse
    A list of queries.
    scriptVariables Map<String,String>
    Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
    continueOnFailure boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jarFileUris string[]
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
    queryFileUri string
    The HCFS URI of the script that contains the Pig queries.
    queryList QueryListResponse
    A list of queries.
    scriptVariables {[key: string]: string}
    Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
    continue_on_failure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jar_file_uris Sequence[str]
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
    query_file_uri str
    The HCFS URI of the script that contains the Pig queries.
    query_list QueryListResponse
    A list of queries.
    script_variables Mapping[str, str]
    Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    properties Map<String>
    Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
    queryFileUri String
    The HCFS URI of the script that contains the Pig queries.
    queryList Property Map
    A list of queries.
    scriptVariables Map<String>
    Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]).

    PrestoJobResponse

    ClientTags List<string>
    Optional. Presto client tags to attach to this query
    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    OutputFormat string
    Optional. The format in which query output will be displayed. See the Presto documentation for supported output formats
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values. Used to set Presto session properties (https://prestodb.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Presto CLI
    QueryFileUri string
    The HCFS URI of the script that contains SQL queries.
    QueryList Pulumi.GoogleNative.Dataproc.V1.Inputs.QueryListResponse
    A list of queries.
    ClientTags []string
    Optional. Presto client tags to attach to this query
    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    OutputFormat string
    Optional. The format in which query output will be displayed. See the Presto documentation for supported output formats
    Properties map[string]string
    Optional. A mapping of property names to values. Used to set Presto session properties (https://prestodb.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Presto CLI
    QueryFileUri string
    The HCFS URI of the script that contains SQL queries.
    QueryList QueryListResponse
    A list of queries.
    clientTags List<String>
    Optional. Presto client tags to attach to this query
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    outputFormat String
    Optional. The format in which query output will be displayed. See the Presto documentation for supported output formats
    properties Map<String,String>
    Optional. A mapping of property names to values. Used to set Presto session properties (https://prestodb.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Presto CLI
    queryFileUri String
    The HCFS URI of the script that contains SQL queries.
    queryList QueryListResponse
    A list of queries.
    clientTags string[]
    Optional. Presto client tags to attach to this query
    continueOnFailure boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    outputFormat string
    Optional. The format in which query output will be displayed. See the Presto documentation for supported output formats
    properties {[key: string]: string}
    Optional. A mapping of property names to values. Used to set Presto session properties (https://prestodb.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Presto CLI
    queryFileUri string
    The HCFS URI of the script that contains SQL queries.
    queryList QueryListResponse
    A list of queries.
    client_tags Sequence[str]
    Optional. Presto client tags to attach to this query
    continue_on_failure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    output_format str
    Optional. The format in which query output will be displayed. See the Presto documentation for supported output formats
    properties Mapping[str, str]
    Optional. A mapping of property names to values. Used to set Presto session properties (https://prestodb.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Presto CLI
    query_file_uri str
    The HCFS URI of the script that contains SQL queries.
    query_list QueryListResponse
    A list of queries.
    clientTags List<String>
    Optional. Presto client tags to attach to this query
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    outputFormat String
    Optional. The format in which query output will be displayed. See the Presto documentation for supported output formats
    properties Map<String>
    Optional. A mapping of property names to values. Used to set Presto session properties (https://prestodb.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Presto CLI
    queryFileUri String
    The HCFS URI of the script that contains SQL queries.
    queryList Property Map
    A list of queries.

    PySparkJobResponse

    ArchiveUris List<string>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    Args List<string>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    FileUris List<string>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    JarFileUris List<string>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainPythonFileUri string
    The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    PythonFileUris List<string>
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    ArchiveUris []string
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    Args []string
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    FileUris []string
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    JarFileUris []string
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainPythonFileUri string
    The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    PythonFileUris []string
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainPythonFileUri String
    The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    pythonFileUris List<String>
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    archiveUris string[]
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args string[]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris string[]
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jarFileUris string[]
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainPythonFileUri string
    The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    pythonFileUris string[]
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    archive_uris Sequence[str]
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args Sequence[str]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    file_uris Sequence[str]
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jar_file_uris Sequence[str]
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    main_python_file_uri str
    The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    python_file_uris Sequence[str]
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    mainPythonFileUri String
    The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    properties Map<String>
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    pythonFileUris List<String>
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.

    QueryListResponse

    Queries List<string>
    The queries to execute. You do not need to end a query expression with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of a Dataproc API snippet that uses a QueryList to specify a HiveJob: "hiveJob": { "queryList": { "queries": [ "query1", "query2", "query3;query4", ] } }
    Queries []string
    The queries to execute. You do not need to end a query expression with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of a Dataproc API snippet that uses a QueryList to specify a HiveJob: "hiveJob": { "queryList": { "queries": [ "query1", "query2", "query3;query4", ] } }
    queries List<String>
    The queries to execute. You do not need to end a query expression with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of a Dataproc API snippet that uses a QueryList to specify a HiveJob: "hiveJob": { "queryList": { "queries": [ "query1", "query2", "query3;query4", ] } }
    queries string[]
    The queries to execute. You do not need to end a query expression with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of a Dataproc API snippet that uses a QueryList to specify a HiveJob: "hiveJob": { "queryList": { "queries": [ "query1", "query2", "query3;query4", ] } }
    queries Sequence[str]
    The queries to execute. You do not need to end a query expression with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of a Dataproc API snippet that uses a QueryList to specify a HiveJob: "hiveJob": { "queryList": { "queries": [ "query1", "query2", "query3;query4", ] } }
    queries List<String>
    The queries to execute. You do not need to end a query expression with a semicolon. Multiple queries can be specified in one string by separating each with a semicolon. Here is an example of a Dataproc API snippet that uses a QueryList to specify a HiveJob: "hiveJob": { "queryList": { "queries": [ "query1", "query2", "query3;query4", ] } }

    SparkJobResponse

    ArchiveUris List<string>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    Args List<string>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    FileUris List<string>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    JarFileUris List<string>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainClass string
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in SparkJob.jar_file_uris.
    MainJarFileUri string
    The HCFS URI of the jar file that contains the main class.
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    ArchiveUris []string
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    Args []string
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    FileUris []string
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    JarFileUris []string
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainClass string
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in SparkJob.jar_file_uris.
    MainJarFileUri string
    The HCFS URI of the jar file that contains the main class.
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainClass String
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in SparkJob.jar_file_uris.
    mainJarFileUri String
    The HCFS URI of the jar file that contains the main class.
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archiveUris string[]
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args string[]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris string[]
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jarFileUris string[]
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainClass string
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in SparkJob.jar_file_uris.
    mainJarFileUri string
    The HCFS URI of the jar file that contains the main class.
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archive_uris Sequence[str]
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args Sequence[str]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    file_uris Sequence[str]
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jar_file_uris Sequence[str]
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    main_class str
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in SparkJob.jar_file_uris.
    main_jar_file_uri str
    The HCFS URI of the jar file that contains the main class.
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver and tasks.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    mainClass String
    The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in SparkJob.jar_file_uris.
    mainJarFileUri String
    The HCFS URI of the jar file that contains the main class.
    properties Map<String>
    Optional. A mapping of property names to values, used to configure Spark. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.

    SparkRJobResponse

    ArchiveUris List<string>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    Args List<string>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    FileUris List<string>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainRFileUri string
    The HCFS URI of the main R file to use as the driver. Must be a .R file.
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure SparkR. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    ArchiveUris []string
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    Args []string
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    FileUris []string
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    MainRFileUri string
    The HCFS URI of the main R file to use as the driver. Must be a .R file.
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure SparkR. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainRFileUri String
    The HCFS URI of the main R file to use as the driver. Must be a .R file.
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure SparkR. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archiveUris string[]
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args string[]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris string[]
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    mainRFileUri string
    The HCFS URI of the main R file to use as the driver. Must be a .R file.
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure SparkR. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archive_uris Sequence[str]
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args Sequence[str]
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    file_uris Sequence[str]
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    main_r_file_uri str
    The HCFS URI of the main R file to use as the driver. Must be a .R file.
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure SparkR. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    archiveUris List<String>
    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
    args List<String>
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    fileUris List<String>
    Optional. HCFS URIs of files to be placed in the working directory of each executor. Useful for naively parallel tasks.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    mainRFileUri String
    The HCFS URI of the main R file to use as the driver. Must be a .R file.
    properties Map<String>
    Optional. A mapping of property names to values, used to configure SparkR. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.

    SparkSqlJobResponse

    JarFileUris List<string>
    Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Dataproc API might be overwritten.
    QueryFileUri string
    The HCFS URI of the script that contains SQL queries.
    QueryList Pulumi.GoogleNative.Dataproc.V1.Inputs.QueryListResponse
    A list of queries.
    ScriptVariables Dictionary<string, string>
    Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
    JarFileUris []string
    Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    Properties map[string]string
    Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Dataproc API might be overwritten.
    QueryFileUri string
    The HCFS URI of the script that contains SQL queries.
    QueryList QueryListResponse
    A list of queries.
    ScriptVariables map[string]string
    Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    properties Map<String,String>
    Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Dataproc API might be overwritten.
    queryFileUri String
    The HCFS URI of the script that contains SQL queries.
    queryList QueryListResponse
    A list of queries.
    scriptVariables Map<String,String>
    Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
    jarFileUris string[]
    Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    properties {[key: string]: string}
    Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Dataproc API might be overwritten.
    queryFileUri string
    The HCFS URI of the script that contains SQL queries.
    queryList QueryListResponse
    A list of queries.
    scriptVariables {[key: string]: string}
    Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
    jar_file_uris Sequence[str]
    Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    properties Mapping[str, str]
    Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Dataproc API might be overwritten.
    query_file_uri str
    The HCFS URI of the script that contains SQL queries.
    query_list QueryListResponse
    A list of queries.
    script_variables Mapping[str, str]
    Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).
    jarFileUris List<String>
    Optional. HCFS URIs of jar files to be added to the Spark CLASSPATH.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    properties Map<String>
    Optional. A mapping of property names to values, used to configure Spark SQL's SparkConf. Properties that conflict with values set by the Dataproc API might be overwritten.
    queryFileUri String
    The HCFS URI of the script that contains SQL queries.
    queryList Property Map
    A list of queries.
    scriptVariables Map<String>
    Optional. Mapping of query variable names to values (equivalent to the Spark SQL command: SET name="value";).

    TrinoJobResponse

    ClientTags List<string>
    Optional. Trino client tags to attach to this query
    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    LoggingConfig Pulumi.GoogleNative.Dataproc.V1.Inputs.LoggingConfigResponse
    Optional. The runtime log config for job execution.
    OutputFormat string
    Optional. The format in which query output will be displayed. See the Trino documentation for supported output formats
    Properties Dictionary<string, string>
    Optional. A mapping of property names to values. Used to set Trino session properties (https://trino.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Trino CLI
    QueryFileUri string
    The HCFS URI of the script that contains SQL queries.
    QueryList Pulumi.GoogleNative.Dataproc.V1.Inputs.QueryListResponse
    A list of queries.
    ClientTags []string
    Optional. Trino client tags to attach to this query
    ContinueOnFailure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    LoggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    OutputFormat string
    Optional. The format in which query output will be displayed. See the Trino documentation for supported output formats
    Properties map[string]string
    Optional. A mapping of property names to values. Used to set Trino session properties (https://trino.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Trino CLI
    QueryFileUri string
    The HCFS URI of the script that contains SQL queries.
    QueryList QueryListResponse
    A list of queries.
    clientTags List<String>
    Optional. Trino client tags to attach to this query
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    outputFormat String
    Optional. The format in which query output will be displayed. See the Trino documentation for supported output formats
    properties Map<String,String>
    Optional. A mapping of property names to values. Used to set Trino session properties (https://trino.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Trino CLI
    queryFileUri String
    The HCFS URI of the script that contains SQL queries.
    queryList QueryListResponse
    A list of queries.
    clientTags string[]
    Optional. Trino client tags to attach to this query
    continueOnFailure boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    loggingConfig LoggingConfigResponse
    Optional. The runtime log config for job execution.
    outputFormat string
    Optional. The format in which query output will be displayed. See the Trino documentation for supported output formats
    properties {[key: string]: string}
    Optional. A mapping of property names to values. Used to set Trino session properties (https://trino.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Trino CLI
    queryFileUri string
    The HCFS URI of the script that contains SQL queries.
    queryList QueryListResponse
    A list of queries.
    client_tags Sequence[str]
    Optional. Trino client tags to attach to this query
    continue_on_failure bool
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    logging_config LoggingConfigResponse
    Optional. The runtime log config for job execution.
    output_format str
    Optional. The format in which query output will be displayed. See the Trino documentation for supported output formats
    properties Mapping[str, str]
    Optional. A mapping of property names to values. Used to set Trino session properties (https://trino.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Trino CLI
    query_file_uri str
    The HCFS URI of the script that contains SQL queries.
    query_list QueryListResponse
    A list of queries.
    clientTags List<String>
    Optional. Trino client tags to attach to this query
    continueOnFailure Boolean
    Optional. Whether to continue executing queries if a query fails. The default value is false. Setting to true can be useful when executing independent parallel queries.
    loggingConfig Property Map
    Optional. The runtime log config for job execution.
    outputFormat String
    Optional. The format in which query output will be displayed. See the Trino documentation for supported output formats
    properties Map<String>
    Optional. A mapping of property names to values. Used to set Trino session properties (https://trino.io/docs/current/sql/set-session.html) Equivalent to using the --session flag in the Trino CLI
    queryFileUri String
    The HCFS URI of the script that contains SQL queries.
    queryList Property Map
    A list of queries.

    YarnApplicationResponse

    Name string
    The application name.
    Progress double
    The numerical progress of the application, from 1 to 100.
    State string
    The application state.
    TrackingUrl string
    Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
    Name string
    The application name.
    Progress float64
    The numerical progress of the application, from 1 to 100.
    State string
    The application state.
    TrackingUrl string
    Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
    name String
    The application name.
    progress Double
    The numerical progress of the application, from 1 to 100.
    state String
    The application state.
    trackingUrl String
    Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
    name string
    The application name.
    progress number
    The numerical progress of the application, from 1 to 100.
    state string
    The application state.
    trackingUrl string
    Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
    name str
    The application name.
    progress float
    The numerical progress of the application, from 1 to 100.
    state str
    The application state.
    tracking_url str
    Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.
    name String
    The application name.
    progress Number
    The numerical progress of the application, from 1 to 100.
    state String
    The application state.
    trackingUrl String
    Optional. The HTTP URL of the ApplicationMaster, HistoryServer, or TimelineServer that provides application-specific information. The URL uses the internal hostname, and requires a proxy server for resolution and, possibly, access.

    Package Details

    Repository
    Google Cloud Native pulumi/pulumi-google-native
    License
    Apache-2.0
    google-native logo

    Google Cloud Native is in preview. Google Cloud Classic is fully supported.

    Google Cloud Native v0.32.0 published on Wednesday, Nov 29, 2023 by Pulumi
    pulumi/pulumi-google-native