You are viewing the RapidMiner Studio documentation for version 9.0 -Check here for latest version
Using the Google Cloud Storage Connector
The Google Cloud Storage Connector allows you to access your Google Cloud Storage directly from RapidMiner Studio. Bothreadandwriteoperations are supported. You can alsoread from a set of filesin a Google Cloud Storage directory, using the
Loop Google Storage操作符。This document will walk you through how to:
Connect to your Google Cloud Storage account
RapidMiner’s use and transfer of information received from Google APIs to any other app will adhere toGoogle API Services User Data Policy, including the Limited Use requirements.
Before you can use the Google Cloud Storage connector, you have to configure a new Google Cloud Storage connection. For this purpose, you will need the connection details of your account. This includes a project id and either an access token, or a private key for a service account.
Open theManage Connectionsdialog in RapidMiner Studio by going to
Connections > Manage Connections.Click onAdd Connection
in the lower left:
Enter a name for the new connection and select
Google Cloud Storage Connectionas theConnection Type:
Fill in the connection details of your Google Storage account. You have two alternative options for that, see next two steps for details.
You may use anaccess tokenthat you get after you allow RapidMiner to access your cloud account on a consent screen. This is the default option: leaveuse service accountunchecked and follow the steps below.
To the right of theaccess tokenfield, click the
button to request an access token.Click onRequest access token
要在您的浏览器中打开谷歌网站。如果你are not already logged into your Google Cloud account, you will have to do so now. You can manually copy the URL by clicking onShow URL instead.
ClickAllowto give RapidMiner access to your Google Cloud account and to generate a token. This will bring you to a page where you can see the access token. Copy the code you get there.
Return to RapidMiner Studio, enter the access token, and click
Complete:
Specify theproject idfor the connection as well.
Alternatively, you may setup aService accountfor your project. In this case, checkuse service accountflag and follow the steps below.
After setting up the Service account, create and download a JSON key for it. Paste the entire JSON file content (e.g. using a text editor and the clipboard) into theprivate key file contentfield.
Specify theproject idfor the connection as well.
While not required, we recommend testing your new Google Cloud Storage connection by clicking the
Testbutton.Click
Save all changesto save your connection and close theManage Connectionswindow. You can now start using the Google Cloud Storage operators.
Connections > Manage Connections.
in the lower left:
button to request an access token.
要在您的浏览器中打开谷歌网站。如果你are not already logged into your Google Cloud account, you will have to do so now. You can manually copy the URL by clicking onShow URL instead.
Complete:
Testbutton.Read from Google Cloud Storage
TheRead Google Storageoperator reads data from your Google Cloud Storage account. The operator can be used to load arbitrary file formats, since it only downloads and does not process the files. To process the files you will need to use additional operators such asRead CSV,Read Excel, orRead XML.
Let us start with reading a simplecsvfile from Google Cloud Storage.
Open a new process
in RapidMiner Studio and chooseBlank Projectfrom the list. Drag theRead Google Storageoperator into theProcessview, and connect its output port to the result port of the process:
Select your Google Cloud Storage connection from theconnectiondrop down menu in theParametersview.
Click on thefilechooser button
to view the files in your Google Cloud Storage account. Select the file that you want to load and clickOpen. Note that you needstorage.buckets.listpermissions on the project to be able to list the buckets and use the file chooser. If you do not have that permission, please type the path from which you want to read from directly into the parameter field.
As mentioned above, the
Read Google Storageoperator does not process the contents of the specified file. In our example, we have chosen acsvfile (acomma separated valuesfile). This file type can be processed via theRead CSV操作符。Add aRead CSVoperator between the
Read Google Storageoperator and the result port. You may set the parameters of theRead CSVoperator - such as column separator -, depending on the format of your csv file:
Run
the process! In theResultsperspective, you should see a table containing the rows and columns of your choosen csv file:
in RapidMiner Studio and chooseBlank Projectfrom the list. Drag the
to view the files in your Google Cloud Storage account. Select the file that you want to load and click
the process! In theResultsperspective, you should see a table containing the rows and columns of your choosen csv file:
You could now use further operators to work with this document, e.g., to determine the commonness of certain events. To write results back to Google Cloud Storage, you can use theWrite Google Storage操作符。It uses the same connection type as theRead Google Storageoperator and has a similar interface. You can alsoread from a set of filesin a Google Cloud Storage directory, using theLoop Google Storage操作符。对于这一点,你需要指定connectionand thefolder, which you want to process, as well the steps of the processing loop with nested operators. For more details read the help of theLoop Google Storage操作符。
