SETUP AND MANAGEMENT OF MODELS IN CUBE CLOUD. Colby M. Brown, AICP PTP

Transcription

1 SETUP AND MANAGEMENT OF MODELS IN CUBE CLOUD Colby M. Brown, AICP PTP 1

2 CUBE CLOUD INTRODUCING CUBE CLOUD Develop the Model with Cube in the Desktop Environment Publish the Model from Cube to the Cube Cloud Create, run and analyze scenarios from anywhere

3 SOFTWARE DEMONSTRATION

4 UNDERSTANDING CUBE CLOUD DIRECTORY STRUCTURE The exporter converts the catalog into a minimal directory structure

5 UNDERSTANDING CUBE CLOUD DIRECTORY STRUCTURE Web Working directory/scenario directory Apps Model script file and catalog information (.xml) Inputs All inputs (scenario specific, fixed inputs) Outputs Empty and used to store fixed outputs when model is run

6 UNDERSTANDING CUBE CLOUD DIRECTORY STRUCTURE Cube Cloud Export Compile model scripts and model catalog Create one model script and a.xml file in web\apps Copy input files to web\inputs Compress web folder for upload Cube Cloud Model Run Each scenario run gets its own copy of the model folder All outputs are saved in the Outputs folder or the Model folder (scenario/working directory)

7 PREPARING FOR CLOUD SOME THINGS TO CONSIDER 7

8 BEST PRACTICES Use Cube Catalog and Application Manager Use file keys for input files Use unique file names for scenario specific input files Reference all model run outputs to the {SCENARIO_DIR} Directly link output files in application manager instead of reading the file internally Always use double quotes for file paths and names Execute user programs from scenario dir

9 INPUT FILE REFERENCES Scenario Specific Input files FILEI input file statements with {SCENARIO_DIR} in path name e.g., FILEI ZONEI[1]= {SCENARIO_DIR}\Zones\ZoneData.DBF Cube Cloud exporter does not recognize scenario specific inputs Warning: Scenario Specific Inputs. The Application scripts are valid, but there is currently no interface to this data in Cube Cloud. Use file keys e.g., FILEI ZONEI[1]= {zonedata}

10 INPUT FILE REFERENCES Fixed Input Files FILEI statements with fixed file path FILEI RECI = C:\Model\Parameters\MC_PAR.DBF Cube Cloud exporter will copy these files to the Inputs folder The Cube Cloud model script will be updated as FILEI RECI = {SCENARIO_DIR}\Inputs\MC_PAR.DBF Cube Cloud user will not be able to update a new version or edit this file Use a file key to input this file

11 INPUT FILE REFERENCES Internal Input Files FILE= statements for READ, RENUMBER, LOOKUP statements. e.g., READ FILE ="{SCENARIO_DIR}\INPUT\ACCESS.PRN Cube Cloud Exporter does not accept any file path keys in FILE= statements. Exporter does not copy this file to the inputs directory. Warning. The following Internal "FILE=" entries are present in the model. They have been associated with the VOYAGER command shown below, and copied to the appropriate folder. Note that for Cube Cloud, the best and safest approach is to avoid use of FILE=,and use an alternative VOYAGER construct if available; e.g. for PRINT commands, use PRINTO=.

12 INPUT FILE REFERENCES Use a user defined key for these types of files or reference these files using {CATALOG_DIR} key. e.g., READ FILE ="{ACCESS}"

13 OUTPUT FILE REFERENCES FILEO, PRINT FILE= and COPY ENDCOPY statements e.g., FILEO MATO[1] = "{SCENARIO_DIR}\OUTPUT\MATRIX\OUTPUT.MAT Cube Cloud model run will output these files to the outputs directory unless the file is output directly to the scenario directory Subsequent steps reading these files internally will fail in Cube Cloud. Output all scenario specific files directly to {SCENARIO_DIR} e.g., FILEO MATO[1] = "{SCENARIO_DIR}\OUTPUT.MAT

14 OUTPUT TO INTERNAL INPUT FILE FILEO/COPY/PRINTO to READ, RENUMBER,LOOKUP (FILE=) statements Program 1-Matrix: FILEO PRINTO[1] = "{SCENARIO_DIR}\READ_OUT.PRN" Program2-Matrix: READ FILE = "{SCENARIO_DIR}\READ_OUT.PRN Cube Cloud Exporter will show a error message Warning: The following Internal files (defined via FILE=) could not be found, so the Applications will not run correctly It is a good practice to directly link output files to inputs in subsequent programs rather than reading internally

15 FILE PATH KEYS FILEI statements with user defined file path keys FILEI MATI[1] = "{LOCATION}\AllWalk.MAT" Cube Cloud exporter does not accept any keys (other than {CATALOG_DIR}) in file PATH of FILEI statements. Error: The following files have Keys in their Path. This construct is incompatible with Cube Cloud. Use the {CATALOG_DIR} key to refer to this file as shown below or use file keys as discussed above FILEI MATI[1] = "{CATALOG_DIR}\Base\Input\AllWalk.MAT"

16 FILE TOKENS File references using tokens e.g., READ FILE = "@CPI_FILE@" Cube Cloud exporter does not give any warnings or errors Cube Cloud model script is updated to read READ FILE = {CATALOG}\OUTPUTS\@CPI_FILE@ Use valid file paths instead of tokens

17 INPUT FILES WITH SIMILAR NAMES Cube Cloud exporter appends a suffix (_cw#) input files with similar names. It also updates the catalog key values for these files with updated file names e.g., {File1} = C:\Model\Base\ZoneData.dbf {File2} = C:\Model\Base\Parameters\ZoneData.dbf Exporter will copy 2 files to the inputs folder with the names ZoneData.dbf & ZoneData_cw1.dbf and update the file key values as follows {File1}= inputs\zonedata.dbf {File2}= inputs\zonedata_cw1.dbf

18 USER PROGRAMS Built-in User Programs e.g., TRCOPY, TRDEL Cube Cloud exporter will show a warning Warning: User Programs currently require some manual intervention to implement in Cube Cloud. Built-in user programs will work without any problems in Cube Cloud

19 USER PROGRAMS User supplied user programs e.g., any.exe files to perform special functions All input files and output files to/from the user program should be referenced to the {SCENARIO_DIR} Update any control files/input files with file paths Execute the user programs from {SCENARIO_DIR}

20 USER PROGRAMS User programs requiring special set-up Certain user programs (e.g., Java Parallel Processing Framework) might require additional folder set-up and special configuration for running Additional set-up might include background processes to support the model run Citilabs will create a new Cube Cloud image to handle these special cases (e.g., CT-RAMP) according to each client s needs

21 SYSTEM COMMANDS Any MS-DOS system commands e.g., COPY, DEL etc. Cube Cloud Exporter will show a warning message Only copy system commands with no wildcards are currently supported by Cube Cloud for file name conversion. It is recommended to copy the individual files using one copy command for each file as shown below. E.g., *COPY {SCENARIO_DIR}\INPUTS\*.csv {SCENARIO_DIR} - Will not work *Copy {SCENARIO_DIR}\INPUTS\File1.csv {SCENARIO_DIR}

22 CUBE CLOUD CLUSTER SETUP Cube Cloud provides scalability of available cluster nodes for each model run ProcessNum/ProcessList for DistributeMultistep and DistributeIntrastep statements should be calculated within the script based on the number of cores available for that run The number of cores available for the model run is input using a scenario key. No need for cluster commands to start and close cluster nodes. e.g., *Cluster Test 1-10 start exit

23 CUBE CLOUD CLUSTER SETUP {CORES}=### PILOT: If ({CORES}>1) ClusterNodes={CORES}-1 Distribute Multistep=T Intrastep=T Else ClusterNodes=1 Distribute Multistep=F Intrastep=F EndIf Intranodes = INT((ClusterNodes-2)/3) Start1=3 End1= Start1 + Intranodes Start2= End1 + 1 End2= Start2 + Intranodes Start3= End2 + 1 End3= Start3 + Intranodes DistributeMultistep ProcessId= Training ProcessNum=1 RUN PGM=MATRIX DistributeIntrastep ProcessId= Training ProcessList=@Start1@-@End1@ ENDRUN DistributeMultistep ProcessId= Training ProcessNum=2 RUN PGM=HIGHWAY DistributeIntrastep ProcessId= Training ProcessList=@Start2@-@End2@ ENDRUN RUN PGM=MATRIX DistributeIntrastep ProcessId= Training ProcessList=@Start3@-@End3@ ENDRUN

24 MULTIPLE APPLICATIONS OUTPUT/INPUT From Application 1 FILEO MATO[3]="{SCENARIO_DIR}\OUTPUT.MAT In Application 2 FILEI MATI[1]="{SCENARIO_DIR}\OUTPUT.MAT Multiple applications are not currently supported in Cube Cloud Cube Cloud Exporter will show a warning message Warning: The following files require a previous Application to be run first in order to create them

25 REPORTS & APPLIER GROUPS Cube Cloud does not currently allow uploading models with reports or more than one applier group Uploading a model with reports will cause an error in the uploading process Cube Cloud has in-built analysis tools to do charts and maps as well as built-in user access control and administration Delete reports and applier groups before uploading

26 MANAGING ACCESS THROUGH ADMIN CONTROL PANEL CUBE CLOUD

27 ANY QUESTIONS? 27