Posts

ssh banners

 This command will output SSH banners from the server as pseudo-terminal is not allocated. #!/bin/bash ssh <user>@<server> << EOF echo 'hi A' EOF In order to remove the banner, we can either change the command as follows: #!/bin/bash ssh <user>@<server> 'exec bash' << EOF echo 'hi A' EOF or easiest way is  touch ~/.hushlogin

Zip file exploration

 in data exploration, when you have a lot of zip files, you need a way to look at the cotents with extracting each zip file. zcat file.log.zip | less unzip -p archive.zip file1.txt | less OR the best way vim archive.zip

VSCode (Not Visual Studio)

  Shortcuts: Ctrl+P - Open a file Ctrl+Alt+F - Search for a file in Explorer Extensions: Remote Explorer - SSH Git Lense - Git integration Data wrangler - Table view of data frames Zip Tools - Exploring zip files in a data set Windsurf / Codium - Auto Completion R-debugger via command line : install.packages("vscDebugger", repos = "https://manuelhentschel.r-universe.dev") # r-debugger vscode extension

Sending email from Python

 Can setup SMTP inside python itself but for better protection, setup sendmail with SMTP as in other post and then: More focused module :  def send_mail ( email , subject , body = "" , dataframe_dict = {}):     html_content = ""     for name , df in dataframe_dict .items():         html_content += f "<h3> { name } </h3>"         html_content += df .to_html( escape = False , classes = 'table table-striped' , table_id = 'dataframe' )         html_content += "<br><br>"     styled_html = f """     <html>     <body>         { body }         <br><br><br><br>         { html_content }     </body>     </html>     """     # Send HTML email using mail command     process = subprocess...

Bloomberg BQL

 BQL is not part of daily limit. So we can use those to get the data even when limit is hit: =@BQL.QUERY("get(" & TEXTJOIN(",", TRUE, B1:E1) & ") for(members('RAY Index'))","cols=7;rows=5919") where cells B1 to E1 have fields as follows: market_cap px_volume 30_day_average_volume_at_time factor_expected_daily_volume

Python Selenium Webdriver sample program

 this is a sample program using selenium to click a button: from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.service import Service from webdriver_manager.chrome import ChromeDriverManager import time # Set up Chrome options chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--headless")  # Run in headless mode (no UI) chrome_options.add_argument("--disable-gpu")  # Disable GPU acceleration for headless mode chrome_options.add_argument("--no-sandbox")  # Disable sandboxing for headless mode chrome_options.add_argument("--disable-dev-shm-usage")  # Fix shared memory issues # Set up ChromeDriver using Service (Selenium 4) service = Service(ChromeDriverManager().install()) # Initialize WebDriver (Chrome) with the path to your ChromeDriver driver = webdriver.Chrome(service=service, options=chrome_options) try:     # Open the website     driver.get("http://omsprodapp:99...

logger.sh

  #!/bin/bash source /home/ $LOGNAME /.env # Ensure at least two arguments are provided if [ " $# " -lt 2 ]; then     echo "Usage: $0 <command> <file_path> [additional_args...]"     exit 1 fi # Extract filename from the second argument filename = $( basename " $2 " ) # Define log directory and log file path log_dir = "/stage/ $LOGNAME /logs/ $filename " log_file = " $log_dir /$( date +%d).log" # Create log directory if it doesn't exist mkdir -p " $log_dir " # Redirect STDOUT and STDERR to the log file (no output on terminal) exec >> " $log_file " 2>&1 echo "Executing " $@ " at ` date `" # Execute the provided command eval " $@ "