Friday, February 21, 2025

Sending email from Python

 Can setup SMTP inside python itself but for better protection, setup sendmail with SMTP as in other post and then:


import subprocess
import os
import argparse
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from email.mime.base import MIMEBase
from email import encoders

def send_file_as_email(file_path, to, subject):
    """Sends an HTML file as the email body and also as an attachment."""
   
    # If subject is None, use the base file name
    if subject is None:
        subject = os.path.basename(file_path)

    # Create the email with mixed content (text + attachments)
    message = MIMEMultipart("mixed")
    message["Subject"] = subject
    message["To"] = to

    # Read HTML content and attach as email body
    with open(file_path, "r") as file:
        html_content = file.read().strip()

    html_body = MIMEText(html_content, "html")
    message.attach(html_body)

    # Attach the file as an attachment
    with open(file_path, "rb") as file:
        attachment = MIMEBase("application", "octet-stream")
        attachment.set_payload(file.read())

    # Encode to base64
    encoders.encode_base64(attachment)
    attachment.add_header("Content-Disposition", f'attachment; filename="{os.path.basename(file_path)}"')
    message.attach(attachment)

    # Send email using sendmail
    process = subprocess.Popen(["/usr/sbin/sendmail", "-t", "-i"], stdin=subprocess.PIPE)
    process.communicate(message.as_bytes())

    print("Email sent successfully!")

def main():
    """Parses command-line arguments and sends an email."""
   
    parser = argparse.ArgumentParser(description="Send an HTML file as an email body & attachment.")
    parser.add_argument("-f", "--file", required=True, help="Path to the HTML file")
    parser.add_argument("-t", "--to", default="harshal.patil@firl.com", help="Recipient email address (default: harshal.patil@firm.com)")
    parser.add_argument("-s", "--subject", default=None, help="Email subject (default: file name)")

    args = parser.parse_args()

    send_file_as_email(args.file, args.to, args.subject)

if __name__ == "__main__":
    main()

Friday, February 14, 2025

Bloomberg BQL

 BQL is not part of daily limit. So we can use those to get the data even when limit is hit:


=@BQL.QUERY("get(" & TEXTJOIN(",", TRUE, B1:E1) & ") for(members('RAY Index'))","cols=7;rows=5919")


where cells B1 to E1 have fields as follows:

market_cap px_volume 30_day_average_volume_at_time factor_expected_daily_volume


Thursday, February 13, 2025

Python Selenium Webdriver sample program

 this is a sample program using selenium to click a button:


from selenium import webdriver

from selenium.webdriver.common.by import By

from selenium.webdriver.chrome.service import Service

from webdriver_manager.chrome import ChromeDriverManager

import time


# Set up Chrome options

chrome_options = webdriver.ChromeOptions()

chrome_options.add_argument("--headless")  # Run in headless mode (no UI)

chrome_options.add_argument("--disable-gpu")  # Disable GPU acceleration for headless mode

chrome_options.add_argument("--no-sandbox")  # Disable sandboxing for headless mode

chrome_options.add_argument("--disable-dev-shm-usage")  # Fix shared memory issues


# Set up ChromeDriver using Service (Selenium 4)

service = Service(ChromeDriverManager().install())


# Initialize WebDriver (Chrome) with the path to your ChromeDriver

driver = webdriver.Chrome(service=service, options=chrome_options)


try:

    # Open the website

    driver.get("http://omsprodapp:9900/oms/f#ListedWrapper")

    

    # Wait for the page to load (you might need to adjust this)

    time.sleep(5)


    # Find the "Blotter All" button by its ID and click it

    blotter_button = driver.find_element(By.ID, "blotAllButt")

    blotter_button.click()


    # Wait for the action to complete (adjust the sleep if needed)

    time.sleep(2)

    print("Successfully clicked 'Blotter All'!")


finally:

    # Close the browser after the task is completed

    driver.quit()


Wednesday, February 12, 2025

logger.sh

 #!/bin/bash

source /home/$LOGNAME/.env

# Ensure at least two arguments are provided
if [ "$#" -lt 2 ]; then
    echo "Usage: $0 <command> <file_path> [additional_args...]"
    exit 1
fi

# Extract filename from the second argument
filename=$(basename "$2")

# Define log directory and log file path
log_dir="/stage/$LOGNAME/logs/$filename"
log_file="$log_dir/$(date +%d).log"

# Create log directory if it doesn't exist
mkdir -p "$log_dir"

# Redirect STDOUT and STDERR to the log file (no output on terminal)
exec >> "$log_file" 2>&1

echo "Executing "$@" at `date`"

# Execute the provided command
eval "$@"

Tuesday, February 11, 2025

Windows 11 Pro cannot grab window when maximized

 Win + R -> "control access.cpl"

Make the mouse easier to use

UnCheck "Prevent windows from being automatically arranged"

Thursday, February 6, 2025

Loading env file in bash

 If you want to load .env file before running a command in bash, we can use following utility. (Can use the just the command as well.)


14:54:00 harshal@HP:~$ set -a; source ~/.env; set +a; CMD


or


14:55:42 harshal@HP:~$ cat ~/load_dot_env.sh

set -a; source ~/.env; set +a

14:55:47 harshal@HP:~$ source ~/load_dot_env.sh 

14:55:49 harshal@HP:~$ 

 



Wednesday, February 5, 2025

Citrix workspace on Ubuntu 2404

Typically, we start Citrix with this command:

/opt/Citrix/ICAClient/selfservice

But I was getting this error :

harsshal@HP:~$ /opt/Citrix/ICAClient/selfservice 

/opt/Citrix/ICAClient/selfservice: error while loading shared libraries: libwebkit2gtk-4.0.so.37: cannot open shared object file: No such file or directory

But followed these steps and then it worked:

sudo add-apt-repository -y "deb http://gb.archive.ubuntu.com/ubuntu jammy main"

sudo apt update

sudo apt install libwebkit2gtk-4.0-dev

To enable middle click, Change the configuration file  :

sudo vim /opt/Citrix/ICAClient/config/All_Regions.ini:

MouseSendsControlV=False 

AutoHotKey programs

volume.ahk

#Requires AutoHotkey v2.0

Volume_Down::SoundSetVolume "-10"

Volume_Up::SoundSetVolume "+10"

Volume_Mute::SoundSetMute "-1"


mouse.ahk


#Requires AutoHotkey v2.0

waitTime := 5 ; Number of minutes to wait


Loop {

    Sleep waitTime * 60 * 1000

    MouseMove 10, 0, 20, "R"

    MouseMove -10, 0, 20, "R"

}

blotter.ahk

#Requires AutoHotkey v2.0  ; Ensure the script runs with AutoHotkey v2

SetTimer RunCommand, 3600000  ; Set a timer to run every 1 hour (3600000 milliseconds)

RunCommand()
{
    currentTime := FormatTime("HH")  ; Corrected FormatTime call
    if ((currentTime >= 7 and currentTime <= 15) or currentTime = 17)  ; Check if it's between 7 AM - 3 PM or exactly 5 PM
    {
        Run("C:/Users/hpatil/AppData/Local/Microsoft/WindowsApps/python.exe c:/Users/hpatil/webdr.py")
    }
}

or

 #Requires AutoHotkey v2.0  ; Ensure the script runs with AutoHotkey v2

SetTimer RunCommand, 60000  ; Run every 60 seconds (1 minute)

RunCommand()

{

    run_times := ["0715", "0815", "1000", "1340", "1500", "1700"]  ; Define allowed times

    currentTime := FormatTime(A_Now, "HHmm")  ; Get current time in HHMM format

    if HasVal(run_times, currentTime)  ; Check if currentTime is in the array

    {

        Run("C:/Users/hpatil/AppData/Local/Microsoft/WindowsApps/python.exe c:/Users/hpatil/webdr.py")

    }

}

HasVal(haystack, needle)

{

    if !IsObject(haystack)  ; Ensure it's an array before looping

        throw Error("Bad haystack!")  ; Use `Error` instead of `Exception`

    for value in haystack

        if (value = needle)

            return true  ; Return true if found

    return false  ; Return false if not found

}



Monday, January 27, 2025

Sending email via command line

Requirements:

sudo apt install mailutils
sudo apt-get install ssmtp

cat /etc/ssmtp/ssmtp.conf
mailhub=email.firm.com:25 <smpt server>


Then there are multiple ways to it 

cat temp
To: harshal.patil@adyne.com 
Subject: Some Notifying Email 
MIME-Version: 1.0 
Content-Type: text/plain 
Body of your email goes here. Hello world

cat temp | sendmail -i -t

OR

 echo -e "To: harshal.patil@firm.com\nSubject: Subject\nContent-type: text/html" | cat - /stage/qspc/projects/sas/tools/sas_slippage_outliers.html | sendmail -i -t

OR

52 07 * * 1-5 echo "To: harshal.patil@firm.com, harshal.patil@firm.com \nSubject: Slippage Outliers $(date +\%F) \nContent-type: text/html\n\n" > /tmp/email_body.txt;cat /stage/qspc/projects/sas/tools/sas_slippage_outliers.html >> /tmp/email_body.txt;cat /tmp/email_body.txt | /usr/sbin/sendmail -i -t > /stage/qspc/logs/sendmail/$(date +\%d).log 2>&1


If wanted to attach as a file (when html contains graphs):

 (cat /stage/qspcadmin/logs/sendmail/intraday_liquidation.header ; uuencode /stage/qspcadmin/projects/sas/tools/intraday_liquidation.html intraday_liquidation.html) | /sbin/sendmail -i -t

where 

$ cat /stage/qspcadmin/logs/sendmail/intraday_liquidation.header
#To: harshal.patil@firm.com, a.b@firm.com
To: harshal.patil@firm.com
Subject: Intraday Strategy Liquidation

if you wanted an image embeded in the bodt:
(
echo "From: sender@example.com"
echo "To: recipient@example.com"
echo "Subject: Embedded Image"
echo "MIME-Version: 1.0"
echo 'Content-Type: multipart/related; boundary="boundary123"'
echo ""
echo "--boundary123"
echo "Content-Type: text/html; charset=UTF-8"
echo ""
echo '<html><body>'
echo '<p>Here is an embedded image:</p>'
echo '<img src="cid:myimage" width="600" height="400">'
echo '</body></html>'
echo ""
echo "--boundary123"
echo "Content-Type: image/jpeg"
echo 'Content-Transfer-Encoding: base64'
echo 'Content-Disposition: inline; filename="image.jpg"'
echo 'Content-ID: <myimage>'
base64 image.jpg
echo "--boundary123--"
) | sendmail -t

Python / pandas last_bday

import pandas as pd

from pandas.tseries.offsets import BDay

today = pd.to_datetime('today')

last_bday = today - BDay(1) 

Thursday, January 23, 2025

Python groupby

 There are 2 ways to normal results:

  • use agg function

grp_df = df.groupby(['BookLevel4','Symbol','Direction','OrderId'], as_index=False).agg({'TxTime':['min','max','count'], 'FillQuantity':['sum']})

grp_df.columns = [''.join(col).strip() for col in grp_df.columns.values]


  • use apply function 
    • In case you use more than 1 column
    • Wont have to normalize multi-level columns


grp_df = df.groupby(['BookLevel4','Symbol','Direction','OrderId'], as_index=False).apply(

        lambda s: pd.Series({

            "TxTimemin": s["TxTime"].min(),

            "TxTimemax": s["TxTime"].max(),

            "TxTimecount": s["TxTime"].count(),

            "FillQuantitysum": s["FillQuantity"].sum(),

            "FillVWAP": (s['FillPrice'] * s['FillQuantity']).sum() / s['FillQuantity'].sum()

        })

    )

Tuesday, January 21, 2025

my .bcrc

 scale=6

my .screenrc

All possible variables are listed here:

https://www.gnu.org/software/screen/manual/html_node/String-Escapes.html#String-Escapes


 Here are the contents of my screenrc:

# Change default scrollback value for new windows

defscrollback 10000 # default: 100


# no welcome message

startup_message off


# advertise hardstatus support to $TERMCAP

termcapinfo  * '' 'hs:ts=\E_:fs=\E\\:ds=\E_\E\\'

termcapinfo xterm* ti@:te@

vbell "off"

bindkey "^[OR" prev

bindkey "^[OS" next


hardstatus alwayslastline

hardstatus string '%{= kG}[%{G}%H%? %1`%?%{g}][%= %{= kw}%-w%{+b yk} %n*%t%?(%u)%? %{-}%+w %=%{g}][%{B}%m/%d %{W}%C%A%{g}]'


screen -t notebook 

screen -t local

Git setup

 Few initial commands to setup git properly

* git config --global user.email "Harshal.Patil@company.com"

* git config --global user.name "Harshal Patil"

* git config --global credential.helper store