Therefore if you plan to reuse this conf to create multiple RDDs, you need to make Parameters: conf - JobConf for setting up the dataset.
file name for a filesystem-based dataset, table name for HyperTable), Get an RDD for a Hadoop-readable dataset from a Hadoop JobConf given its InputFormat and other clearJobGroup public void clearJobGroup().Parameters: groupId - (undocumented) description - (undocumented) interruptOnCancel - (undocumented) Where HDFS may respond to Thread.interrupt() by marking nodes as dead. That the tasks are actually stopped in a timely manner, but is off by default due to HDFS-1208, In Thread.interrupt() being called on the job's executor threads. If interruptOnCancel is set to true for the job group, then job cancellation will result Sc.setJobGroup("some_job_to_cancel", "some job description") The application can also use .cancelJobGroup to cancel all Once set, the Spark web UI will associate such jobs with this group. Often, a unit of execution in an application consists of multiple Spark actions or jobs.Īpplication programmers can use this method to group all those jobs together and give a initializeLogIfNecessary protected static void initializeLogIfNecessary(boolean isInterpreter)Īssigns a group ID to all the jobs started by this thread until the group ID is set to a.isTraceEnabled protected static boolean isTraceEnabled().logError protected static void logError(scala.Function0 msg,.logWarning protected static void logWarning(scala.Function0 msg,.logTrace protected static void logTrace(scala.Function0 msg,.logDebug protected static void logDebug(scala.Function0 msg,.logInfo protected static void logInfo(scala.Function0 msg,.logError protected static void logError(scala.Function0 msg).logWarning protected static void logWarning(scala.Function0 msg).logTrace protected static void logTrace(scala.Function0 msg).logDebug protected static void logDebug(scala.Function0 msg).logInfo protected static void logInfo(scala.Function0 msg).Parameters: obj - (undocumented) Returns: (undocumented) In most cases you can call jarOfObject(this) in Protected įind the JAR that contains the class of a particular object, to make it easy for users Create and register a list accumulator, which starts with empty list and accumulates inputs
0 kommentar(er)
