public final class WriteBigQuery extends CompositeOperator implements RecordSinkOperator
setRetainCloudStoreTempFiles(boolean)
to control this behavior.
Only string and numeric types are supported by BigQuery. All other data types are converted into strings and stored as such. This includes dates and time stamps.
Before using this operator, an account with Google services must be established. The account has to include access to the Google Cloud Storage and Google BigQuery services. Credentials for the account are passed as a property to the operator.
Constructor and Description |
---|
WriteBigQuery() |
Modifier and Type | Method and Description |
---|---|
protected void |
compose(CompositionContext ctx)
Compose the body of this operator.
|
QualifiedTableID |
getBqTableId()
Get the identifier of the target BigQuery table.
|
CredentialInfo |
getCredentials()
Get the credentials used to access Google services.
|
String |
getCsBucketId()
Get the identifier of the Google Cloud Storage bucket to use for
intermediate data storage.
|
RecordPort |
getInput()
Get the input record port of this operator.
|
int |
getMaxBadRecords()
Get the maximum number of bad records that will be skipped before the operation is aborted.
|
long |
getMaxFileSize()
Get the maximum file size of intermediate files written to Google Cloud Storage.
|
WriteMode |
getWriteMode()
Get the write mode that specifies how to write data to the target table.
|
boolean |
isRetainCloudStoreTempFiles()
Return the property that determines if temporary storage space is removed or retained.
|
void |
setBqTableId(QualifiedTableID tableId)
Set the identifier of the Google BigQuery table used to store the input data.
|
void |
setCredentials(CredentialInfo credentials)
Set the credentials used to access the required Google services.
|
void |
setCsBucketId(String csBucketId)
Set the identifier of the Google Cloud Storage bucket to use for
intermediate data storage.
|
void |
setMaxBadRecords(int maxBadRecords)
Sets the maximum number of bad records that will be tolerated before the upload into
the BigQuery table is aborted.
|
void |
setMaxFileSize(long maxChunkSize)
Sets the maximum file size of intermediate files written to Google Cloud Storage.
|
void |
setRetainCloudStoreTempFiles(boolean retainCloudStoreTempFiles)
Sets the property that determines if temporary storage space is removed or retained.
|
void |
setWriteMode(WriteMode writeMode)
Set the write mode specifying how to write data to the target table.
|
disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
disableParallelism, getInputPorts, getOutputPorts
public CredentialInfo getCredentials()
public void setCredentials(CredentialInfo credentials)
credentials
- Google services credentialspublic String getCsBucketId()
public void setCsBucketId(String csBucketId)
csBucketId
- bucket identifierpublic QualifiedTableID getBqTableId()
public void setBqTableId(QualifiedTableID tableId)
tableId
- table identifierpublic void setWriteMode(WriteMode writeMode)
writeMode
- write mode for the target tablepublic WriteMode getWriteMode()
public boolean isRetainCloudStoreTempFiles()
public void setRetainCloudStoreTempFiles(boolean retainCloudStoreTempFiles)
retainCloudStoreTempFiles
- true implies retain; false implies remove (the default)public int getMaxBadRecords()
public void setMaxBadRecords(int maxBadRecords)
maxBadRecords
- maximum bad record countpublic long getMaxFileSize()
public void setMaxFileSize(long maxChunkSize)
maxChunkSize
- maximum file size in bytespublic RecordPort getInput()
getInput
in interface RecordSinkOperator
getInput
in interface SinkOperator<RecordPort>
protected void compose(CompositionContext ctx)
CompositeOperator
OperatorComposable.add(O)
OperatorComposable.connect(P, P)
. This includes
connections from the composite's input ports to sub-operators, connections between sub-operators, and
connections from sub-operators output ports to the composite's output portscompose
in class CompositeOperator
ctx
- the contextCopyright © 2015 Actian Corporation. All Rights Reserved.