create a new file containing everything but our single line commands and empty lines. Grep has a -v parameter which selects everything but the pattern specified: ________________________________________________________ grep -v “^COMMENT ON”|grep -v “CREATE S”|grep -v “ALTER TABLE”|grep -v “^$” >/sql/multi-line-commands.sql ________________________________________________________ Separate out the different SQL commands into separate files using the Linux csplit command For Ubuntu: ________________________________________________________ csplit --prefix vertica_object --quiet --elide-empty-files multi-line-commands.sql '/^CREATE/' '{*}' -b '%03d' ________________________________________________________ For Mac OSX and Centos: ________________________________________________________ Csplit -n4 ks -f vertica_object multi-line_commands.sql /^CREATE/ {5000} ________________________________________________________ This will produce a file per individual object • vertica_object_0000 • vertica_object_0001 • … • vertica_object_999 Next, identify which object and object type each vertica_object_xxx file represents. To iterate over the files: ________________________________________________________ for f in ./sql/vertica_object* do shell/rename_csplit_files.sh $f done ________________________________________________________ The rename_csplit_files.sh script: ________________________________________________________ #!/bin/bash read -r firstline <$1 echo $firstline |awk -F ' ' -v file_name=$1 ./awk/csplit_rename.awk >>./shell/generate_rename.sh ________________________________________________________ The first line of our vertica_object_xxx file contains the CREATE TABLE, CREATE VIEW or CREATE PROJECTION command and the name of the object it is trying to create. Submit that line to our awk program csplit_rename.awk, which splits the first line using the space characters as the delimiter. $1 $2 $3 $4 CREATE TABLE {schema.table} CREATE TEMPORARY TABLE {schema.table} CREATE FLEX TABLE {schema.table} CREATE FLEXIBLE TABLE {schema.table} CREATE VIEW {schema.view} CREATE PROJECTION {schema.projection} The csplit_rename.awk file: ________________________________________________________ function ltrim(s) {sub(/^[ \t\r\n]+/, “”, s);return s } function ltrim(s) {sub(/[ \t\r\n]+$/, “”, s);return s } function trim(s){return rtrim(ltrim(s));} { tmp_type_name = trim(tolower($2)) if (tmp_type_name == “temporary” || tmp_type_name == “flex”|| tmp_type_name == “flexible”) { object_name = trim($4) object_type_name = “table” } else if (tmp_type_name == “function”) { split(trim($3), object_name_part, “(“) object_name = object_name_part[1] object_type_name = tmp_type_name } else { object_name = trim($3) object_type_name = tmp_type_name } if(object_name>””){ printf “mv %s ./sql/%s/%s..sql\n”, trim(file_name),object_type_name, object_name } } ________________________________________________________ The resulting generate_rename.sh file: ________________________________________________________ mv ./sql/vertica_object_000 ./sql/table/staging.useragent.sql mv ./sql/vertica_object_001 ./sql/view/staging.useragent_redacted.sql ________________________________________________________ When the generate_rename.sh script is run all the vertica_object files are correctly named and moved into their relevant folders.