Bash script to migrate CONTENT TAF's to LIBRARY

Hello Guys,

i want to move my old CONTENT TAF’s to the new LIBRARY schema. Due to i have lots of TAF’s i wrote a smal bash script, which does the job for me. Maybe it will help somebody else, too.

grafik

What it does?

  1. It will go through all folders in CONTENT directory.
  2. If TAF with name ‘500304E0’ exists, it extracts the AudioID from TAF using opus2tonie.py.
  3. If file is not already in LIBRARY folder, it will be copied and renamed there and deleted from CONTENT.
  4. Afterwards the 'source" field is linked to the new location inside the JSON file.
  5. If ‘500304E0’ is not found, it will be validated, if a linked source exists. If not exitst(→missing) is written to console.
  6. Using ./_move2lib.sh -t switch will run in “Test only” mode which does not move or change anything.

How to use it?

  1. Create new file and copy code.
  2. Grant file execution permissions
  3. Adjust the follwoing three variables to your setup:
    CONENT → TeddyCloud content directory, e.g. “/teddycloud/content/default/”
    LIBRARY→ TeddyCloud library directory, e.g. “/teddycloud/library/by/audioID/”
    OPUS2TONIE → path to Opus2Tonie python script which is used to extract the AudioID from TAF

!!!PLEASE MAKE A BACKUP OF YOUR CONTENT AND LIBRARY DIRECTORY BEFORE USING THIS SCRIPT!!!

Prerequisites

  1. Opus2Tonie must be downloaded. It requires Python3
  2. Opus2Tonie requires python3-protobuf to decoce protobuf. Please install it via pip or apt

Bash script

./_move2lib.sh -t

#!/bin/bash

# PLEASE ADJUST HERE TO YOUR ENVIRONMENT!
OPUS2TONIE="/srv/Transfer/TAF_CONVERTER/opus2tonie-main/opus2tonie.py"
CONTENT="/teddycloud/content/default/"
LIBRARY="/teddycloud/library/by/audioID/"

# Init
VERSION=1.0
TEST=0
FORCE=0
SPLIT=1

echo
echo -e "\e[1mMigrate CONTENT TAF's to LIBRARY  v$VERSION\e[0m"
echo

# Check environment
[ ! -d $CONTNET ] && exit 101
[ ! -d $LIBRARY ] && exit 102
[ ! -f $OPUS2TONIE] && exit 103

while getopts "h?t" opt; do
  case "$opt" in
    t )
       TEST=1
        ;;
    h|\?|* )
        usage
        ;;
  esac
done


for DIR in $CONTENT*; do
  PROCESS=0
  VALIDATE=0
  echo -en "Processing dir \"$DIR\": "

  # Check if TAF was already moved to LIBRARY
  if [ -f "${DIR}/500304E0" ] ; then                                                            PROCESS=1; fi
  if [ -f "${DIR}/500304E0.json" ] && [ ! -f "${DIR}/500304E0" ] ; then VALIDATE=1; fi

  if [ $VALIDATE -eq "1" ] ; then
    SOURCE=$(cat ${DIR}/500304E0.json | grep source | awk -F '/' '{print $5}' | awk -F '"' '{print $1}')
        if [ ! -f ${LIBRARY}${SOURCE} ] ; then
      echo -e "\e[31mTAF not found!\e[0m"
      continue
        fi
  fi

  if [ ! -f "${DIR}/500304E0" ] ; then
    echo -e "\e[34mDone\e[0m."
        continue
  elif [ ! -f "${DIR}/500304E0.json" ] ; then
    PROCESS=0
    echo -e "\e[33mJSON not found\e[0m."
    continue
  fi

  if [ $PROCESS -eq "1" ] ; then
    # Extraxt AUDIO_ID from TAF. Use date command with UTC time!
    DATE_TIME=$($OPUS2TONIE --info ${DIR}/500304E0 | grep Timestamp | awk -F ']' '{print $3}' | xargs)
    AUDIO_ID=$(date -u --date "$DATE_TIME" +"%s")

    # Indicate when TAF already exists (e.g. Custom Tonie linked to original)
    [ -f ${LIBRARY}${AUDIO_ID}.taf ] && echo -en "!"

    if [ $TEST -eq "0" ] ; then
      # Copy TAF to Library
      cp -n ${DIR}/500304E0 ${LIBRARY}${AUDIO_ID}.taf
      # Link source to new location
      sed -i "s|\"source\":     \"\",|\"source\":       \"lib://by/audioID/$AUDIO_ID.taf\",|g" "${DIR}/500304E0.json"
      #remove TAF if found at new location
      [ -f ${LIBRARY}${AUDIO_ID}.taf ] && rm ${DIR}/500304E0
    else
      echo -en "% "
        fi
    echo -e "\e[32mOk\e[0m."
  fi
done
exit 0

Hope this wil help.

Cheers.

Why not using the existing migration API within teddyCloud?

HOST=<…>
PORT=7780
CUSTOM_DEFAULT=<path-to-custom/default>
cd

for d in *; do ls $CUSTOM_DEFAULT/$d && echo $d && cp $d/500304E0 $CUSTOM_DEFAULT/$d && curl -X POST -d “ruid=${d}500304E0&libroot=false” “http://$HOST:$PORT/api/migrateContent2Lib”; done;

2 Likes

Thanks for this snipplet. Didn’t know that is already implemented. No hints in changelog as far i can see.
Last comment only says that you will work on it :slight_smile:

For me the job is done. Thanks for your great work!

1 Like

Hi @0xbadbee,
i don’t get it at all, i tried some changes to adapt to my docker-instance.

CUSTOM_CONTENT is ‘teddycloud_content/_data/default’, or ‘teddycloud_library/_data’ ?
In which directory do you call this script ?

i understand the script at all, but: Where should the renamed TAF be, if you call the api ?

Could you make this clear, please?

Thx in advance,
Ka

Please use the teddyCloud GUI for migration, as there is a little transport icon in the CONTENT section of the Tonies for each file that can be migrated to the library.

Hi,
ok, got it - the files in their directories (TAF-Files), i only looked at the content-root (with all the directories of the tonies) because the dirs already have the “tonie”-icon and i didn’t look any further.

What’s the advantage migrating it to the library “by AudioId” ? Having them all directly without directories ?

by audio id places the content into the library/by/audioId/ folder like it happens when you place a new tonie.

root just places the file in the root folder

And the main advantage of migrating the files to the library is, you can assign the content to other tonies/tags. Only files in the library can be chosen in the gui.