Posts

Showing posts from 2025

Getting external facing IP

 * curl ifconfig.me

Generating your own certificate

* openssl req -newkey rsa:2048 -nodes -keyout myCompany.key -out myCompany.csr * openssl x509 -signkey myCompany.key -in myCompany.csr -req -days 365 -out myCompany.crt

Get versions of python modules installed

 pip freeze | egrep 'pandas|pandas_market_calendars' if you want to install a specific version: pip install pandas_market_calendars==4.5.1

ssh banners

 This command will output SSH banners from the server as pseudo-terminal is not allocated. #!/bin/bash ssh <user>@<server> << EOF echo 'hi A' EOF In order to remove the banner, we can either change the command as follows: #!/bin/bash ssh <user>@<server> 'exec bash' << EOF echo 'hi A' EOF or easiest way is  touch ~/.hushlogin

Zip file exploration

 in data exploration, when you have a lot of zip files, you need a way to look at the cotents with extracting each zip file. zcat file.log.zip | less unzip -p archive.zip file1.txt | less OR the best way vim archive.zip

VSCode (Not Visual Studio)

  Shortcuts: Ctrl+P - Open a file Ctrl+Alt+F - Search for a file in Explorer Extensions: Remote Explorer - SSH Git Lense - Git integration Data wrangler - Table view of data frames Zip Tools - Exploring zip files in a data set Windsurf / Codium - Auto Completion R-debugger via command line : install.packages("vscDebugger", repos = "https://manuelhentschel.r-universe.dev") # r-debugger vscode extension

Sending email from Python

 Can setup SMTP inside python itself but for better protection, setup sendmail with SMTP as in other post and then: More focused module :  def send_mail ( email , subject , body = "" , dataframe_dict = {}):     html_content = ""     for name , df in dataframe_dict .items():         html_content += f "<h3> { name } </h3>"         html_content += df .to_html( escape = False , classes = 'table table-striped' , table_id = 'dataframe' )         html_content += "<br><br>"     styled_html = f """     <html>     <body>         { body }         <br><br><br><br>         { html_content }     </body>     </html>     """     # Send HTML email using mail command     process = subprocess...

Bloomberg BQL

 BQL is not part of daily limit. So we can use those to get the data even when limit is hit: =@BQL.QUERY("get(" & TEXTJOIN(",", TRUE, B1:E1) & ") for(members('RAY Index'))","cols=7;rows=5919") where cells B1 to E1 have fields as follows: market_cap px_volume 30_day_average_volume_at_time factor_expected_daily_volume

Python Selenium Webdriver sample program

 this is a sample program using selenium to click a button: from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.chrome.service import Service from webdriver_manager.chrome import ChromeDriverManager import time # Set up Chrome options chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--headless")  # Run in headless mode (no UI) chrome_options.add_argument("--disable-gpu")  # Disable GPU acceleration for headless mode chrome_options.add_argument("--no-sandbox")  # Disable sandboxing for headless mode chrome_options.add_argument("--disable-dev-shm-usage")  # Fix shared memory issues # Set up ChromeDriver using Service (Selenium 4) service = Service(ChromeDriverManager().install()) # Initialize WebDriver (Chrome) with the path to your ChromeDriver driver = webdriver.Chrome(service=service, options=chrome_options) try:     # Open the website     driver.get("http://omsprodapp:99...

logger.sh

  #!/bin/bash source /home/ $LOGNAME /.env # Ensure at least two arguments are provided if [ " $# " -lt 2 ]; then     echo "Usage: $0 <command> <file_path> [additional_args...]"     exit 1 fi # Extract filename from the second argument filename = $( basename " $2 " ) # Define log directory and log file path log_dir = "/stage/ $LOGNAME /logs/ $filename " log_file = " $log_dir /$( date +%d).log" # Create log directory if it doesn't exist mkdir -p " $log_dir " # Redirect STDOUT and STDERR to the log file (no output on terminal) exec >> " $log_file " 2>&1 echo "Executing " $@ " at ` date `" # Execute the provided command eval " $@ "

Windows 11 Pro cannot grab window when maximized

 Win + R -> "control access.cpl" Make the mouse easier to use UnCheck "Prevent windows from being automatically arranged"

Loading env file in bash

 If you want to load .env file before running a command in bash, we can use following utility. (Can use the just the command as well.) 14:54:00 harshal@HP:~$ set -a; source ~/.env; set +a; CMD or 14:55:42 harshal@HP:~$ cat ~/load_dot_env.sh set -a; source ~/.env; set +a 14:55:47 harshal@HP:~$ source ~/load_dot_env.sh  14:55:49 harshal@HP:~$   

Citrix workspace on Ubuntu 2404

Typically, we start Citrix with this command: /opt/Citrix/ICAClient/selfservice But I was getting this error : harsshal@HP:~$ /opt/Citrix/ICAClient/selfservice  /opt/Citrix/ICAClient/selfservice: error while loading shared libraries: libwebkit2gtk-4.0.so.37: cannot open shared object file: No such file or directory But followed these steps and then it worked: sudo add-apt-repository -y "deb http://gb.archive.ubuntu.com/ubuntu jammy main" sudo apt update sudo apt install libwebkit2gtk-4.0-dev To enable middle click, Change the configuration file  : sudo vim /opt/Citrix/ICAClient/config/All_Regions.ini: MouseSendsControlV=False 

AutoHotKey programs

volume.ahk #Requires AutoHotkey v2.0 Volume_Down::SoundSetVolume "-10" Volume_Up::SoundSetVolume "+10" Volume_Mute::SoundSetMute "-1" mouse.ahk #Requires AutoHotkey v2.0 waitTime := 5 ; Number of minutes to wait Loop {     Sleep waitTime * 60 * 1000     MouseMove 10, 0, 20, "R"     MouseMove -10, 0, 20, "R" } blotter.ahk #Requires AutoHotkey v2.0  ; Ensure the script runs with AutoHotkey v2 SetTimer RunCommand, 3600000  ; Set a timer to run every 1 hour (3600000 milliseconds) RunCommand() {     currentTime := FormatTime("HH")  ; Corrected FormatTime call     if ((currentTime >= 7 and currentTime <= 15) or currentTime = 17)  ; Check if it's between 7 AM - 3 PM or exactly 5 PM     {         Run("C:/Users/hpatil/AppData/Local/Microsoft/WindowsApps/python.exe c:/Users/hpatil/webdr.py")     } } or  #Requires AutoHotkey v2.0  ; Ensure the script runs with AutoHotkey v2...

Sending email via command line

Requirements: sudo apt install mailutils sudo apt-get install ssmtp cat /etc/ssmtp/ssmtp.conf mailhub=email.firm.com:25 <smpt server> Then there are multiple ways to it  cat temp To: harshal.patil@adyne.com  Subject: Some Notifying Email  MIME-Version: 1.0  Content-Type: text/plain  Body of your email goes here. Hello world cat temp | sendmail -i -t OR  echo -e "To: harshal.patil@firm.com\nSubject: Subject\nContent-type: text/html" | cat - /stage/qspc/projects/sas/tools/sas_slippage_outliers.html | sendmail -i -t OR 52 07 * * 1-5 echo "To: harshal.patil@firm.com, harshal.patil@firm.com \nSubject: Slippage Outliers $(date +\%F) \nContent-type: text/html\n\n" > /tmp/email_body.txt;cat /stage/qspc/projects/sas/tools/sas_slippage_outliers.html >> /tmp/email_body.txt;cat /tmp/email_body.txt | /usr/sbin/sendmail -i -t > /stage/qspc/logs/sendmail/$(date +\%d).log 2>&1 If wanted to attach as a file (when html contains graphs):  (cat /...

Python / pandas last_bday

import pandas as pd from pandas.tseries.offsets import BDay today = pd.to_datetime('today') last_bday = today - BDay(1) 

Python groupby

 There are 2 ways to normal results: use agg function grp_df = df.groupby(['BookLevel4','Symbol','Direction','OrderId'], as_index=False).agg({'TxTime':['min','max','count'], 'FillQuantity':['sum']}) grp_df.columns = [''.join(col).strip() for col in grp_df.columns.values] use apply function  In case you use more than 1 column Wont have to normalize multi-level columns grp_df = df.groupby(['BookLevel4','Symbol','Direction','OrderId'], as_index=False).apply(         lambda s: pd.Series({             "TxTimemin": s["TxTime"].min(),             "TxTimemax": s["TxTime"].max(),             "TxTimecount": s["TxTime"].count(),             "FillQuantitysum": s["FillQuantity"].sum(),             "FillVWAP": (s['FillPrice'] * s['FillQuantity']).sum() / s['FillQuantity...

my .bcrc

 scale=6

my .screenrc

All possible variables are listed here: https://www.gnu.org/software/screen/manual/html_node/String-Escapes.html#String-Escapes  Here are the contents of my screenrc: # Change default scrollback value for new windows defscrollback 10000 # default: 100 # no welcome message startup_message off # advertise hardstatus support to $TERMCAP termcapinfo  * '' 'hs:ts=\E_:fs=\E\\:ds=\E_\E\\' termcapinfo xterm* ti@:te@ vbell "off" bindkey "^[OR" prev bindkey "^[OS" next hardstatus alwayslastline hardstatus string '%{= kG}[%{G}%H%? %1`%?%{g}][%= %{= kw}%-w%{+b yk} %n*%t%?(%u)%? %{-}%+w %=%{g}][%{B}%m/%d %{W}%C%A%{g}]' screen -t notebook  screen -t local