ZSH history file without duplicates

I made a hack for ZSH which I borrowed for BASH

My history settings .zshrc:

HISTFILE=~/.zhistory
SAVEHIST=10000
HISTSIZE=10000

## Navigate to directories without ls
setopt autocd
 
## Typo correction
setopt CORRECT
 
## Addition of the history file
setopt APPEND_HISTORY
 
## Ignore all repetitions of commands
setopt HIST_IGNORE_ALL_DUPS

## Do not display the string found earlier
setopt HIST_FIND_NO_DUPS

## Ignore rows if they are duplicates
setopt HIST_IGNORE_DUPS
 
## Delete empty lines from history file
setopt HIST_REDUCE_BLANKS

## Ignore a record starting with a space
setopt HIST_IGNORE_SPACE

## Do not add history and fc commands to the history
setopt HIST_NO_STORE
 
## Remove addition of executable files by = cmd
unsetopt EQUALS

The hack № 1:

## Delete duplicates in history
preexec() {
   echo "$(history 0 | sort -k2 -k1nr | \
   uniq -f1 | sort -n | cut -c8-)" > $HISTFILE
}

The hack № 2:

disable r
alias r=' reset'
hf() {
  echo "$(history 1 | sort -k2 -k1nr | \
  uniq -f1 | sort -n | cut -c8-)" > $HISTFILE
  BUFFER="r"; zle accept-line
}; zle -N hf; bindkey '^[[Z' hf # Shift+Tab

To prevent the r alias reset command from getting into history, use the parameter:

## Ignore a record starting with a space
setopt HIST_IGNORE_SPACE

Duplicates in the history file are deleted. The duplicate command drops to the bottom of the file. The chronology of the teams is maintained.

Do not forget to make a copy of the history file when you try. It works for me as I intended, but you may lose the necessary commands in history. Do not be afraid to experiment. :wink:

setopt HIST_IGNORE_DUPS

Always be afraid of re-inventing the wheel :smiley:

It already does what you want:

eg. issue command cccc

2921  zshc
 2924  bbbb
 2926  aaaa
 2928  cccc

isue command aaaa

2921  zshc
 2924  bbbb
 2928  cccc
 2930  aaaa

issue cccc again:

2921  zshc
 2924  bbbb
 2930  aaaa
 2932  cccc
2 Likes

If you carefully read my post, you will see what I have in .zshrc

setopt HIST_IGNORE_DUPS

And I get this without hacking in the history file after closing the terminal:

 2921  zshc
 2924  bbbb
 2928  cccc
 2930  aaaa
 2931  cccc

Perhaps this is because I have SHELL by default BASH.
And I start ZSH in the terminal as a login shell. But not sure if this is because of this.

Do you have zsh or oh my zsh?
Can show all history settings in your .zshrc?

I don’t know why it doesn’t work for me.

Ah, I use zsh as my default, I did see that in your post - I copied it from there :smiley:
Couldn't understand why it didn't work. Why haven't you set your shell to zsh?

Unfortunately, installing ZSH by default did not solve the problem, duplicates are created anyway.

If you work in a session before closing the terminal, then everything works as it should, but if you enter the command -> close the terminal -> open and enter the duplicate, then it will be written in the history.

w
screenruler
w
screenfetch
screenruler

Can show all settings in your .zshrc?

ttyctl -f
stty icrnl
OPENER=mimeo
VISUAL=micro
export "VISUAL_EDITOR"=micro
export EDITOR
#export XDG_CONFIG_HOME=/path/to/nowhere

plugins=(autojump alias-tips)

autoload -U promptinit; promptinit
autoload -Uz compinit
autoload -U mapfile
zmodload zsh/mapfile    
compinit

source /usr/bin/goto

source /usr/share/zsh/plugins/alias-tips/alias-tips.plugin.zsh
source $HOME/.oh-my-zsh/custom/plugins/zsh-histdb/sqlite-history.zsh
autoload -Uz add-zsh-hook
add-zsh-hook precmd histdb-update-outcome

source /home/steve/.oh-my-zsh/custom/plugins/zsh-histdb/histdb-interactive.zsh 
bindkey '^r' _histdb-isearch

source /usr/share/zsh/plugins/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
source /usr/lib/spaceship-prompt/spaceship.zsh
#source ~/.local/bin/tmuxinator.zsh
prompt spaceship

SAVEHIST=10
HISTFILE=~/.zsh_history
setopt HIST_IGNORE_DUPS
setopt HIST_EXPIRE_DUPS_FIRST
setopt HIST_IGNORE_ALL_DUPS

ZSH=/usr/share/oh-my-zsh

DISABLE_AUTO_UPDATE="true"

#BROWSER=/usr/bin/firefox

export POWERLINE_HIDE_USER_NAME=1
export POWERLINE_PATH="short"
export MICRO_TRUECOLOR=1
ZSH_CACHE_DIR=$HOME/.oh-my-zsh-cache

if [[ ! -d $ZSH_CACHE_DIR ]]; then
    mkdir $ZSH_CACHE_DIR
fi

w3mimg () { w3m -o imgdisplay=/usr/lib/w3m/w3mimgdisplay $1}

source /usr/share/oh-my-zsh/oh-my-zsh.sh
source /usr/share/zsh/plugins/zsh-autosuggestions/zsh-autosuggestions.zsh

ALIASFILE=~/.aliasesrc
source $ALIASFILE

function add_alias() {
    if [[ -z $1 || -z $2 || $# -gt 2 ]]; then
        echo usage:
        echo "\t\$$0 ll 'ls -l'"
    else
        echo "alias $1='$2'" >> $ALIASFILE
        echo "alias ADDED to $ALIASFILE"
    fi
    source $ALIASFILE
}
export NVM_DIR="/home/steve/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm

mdcd () {
    mkdir "$1"
    cd "$1"
}

catl () {
    echo "########################################"
    Z=`locate $1 | head -n1`
    echo $Z
    echo "########################################"
    echo .
    cat $Z
}

change () {
    pushd
    bsp
    echo -n "Update Git? "
    local temp
    read temp
    if [[ $temp = "y" || $temp = "Y" ]]; then
        ~/.config/bspwm/gitup.sh
    fi
    popd
}

alias lll="tyls "

function fkill {
    local pid
    pid=$(ps -ef | sed 1d | fzf -e -m -i +s --reverse --tac --margin=4%,1%,1%,2% --inline-info --header="TAB to (un)select. ENTER to kill selected process(es). ESC or CTRL+C to quit." --prompt='Enter string to filter history > ' | awk '{print $2}')

    if [ "x$pid" != "x" ]
    then
        echo $pid | xargs kill -${1:-9}
    fi
}

# fh - repeat history
fh() {
    print -z $( ([ -n "$ZSH_NAME" ] && fc -l 1 || history) | fzf -e +s --tac | sed 's/ *[0-9]* *//' | sed 's/ *[*]* *//')
}

# hdb - repeat history
hdb() {
    print -z $( ([ -n "$ZSH_NAME" ] && histdb --limit 10000 || history) | fzf -e +s --tac |  cut -c 42-)
}



fd() {
    DIR=`find * -maxdepth 0 -type d -print 2> /dev/null | fzf-tmux --reverse` \
        && cd "$DIR"
}

fda() {
    local dir
    dir=$(find ${1:-.} -type d 2> /dev/null | fzf +m --reverse -e +s) && cd "$dir"
}

falias() {
    cat .aliasesrc | fzf -e -1 --reverse
}

alias sxs="pkill -USR1 -x sxhkd"

autoload -U promptinit; promptinit
prompt spaceship
SPACESHIP_PROMPT_ADD_NEWLINE=false
SPACESHIP_PROMPT_SEPARATE_LINE=false
SPACESHIP_PROMPT_DEFAULT_PREFIX=
SPACESHIP_TIME_SHOW=true
SPACESHIP_DIR_PREFIX=$SPACESHIP_CHAR_SYMBOL
SPACESHIP_EXEC_TIME_ELAPSED=10
SPACESHIP_DIR_COLOR=093
#SPACESHIP_TIME_COLOR=red
SPACESHIP_GIT_SHOW=false
SPACESHIP_NODE_SHOW=false


[[ -r "/usr/share/z/z.sh" ]] && source /usr/share/z/z.sh

#unalias z
z() {
    if [[ -z "$*" ]]; then
        cd "$(_z -l 2>&1 | fzf +s --tac | sed 's/^[0-9,.]* *//')"
    else
        _last_z_args="$@"
        _z "$@"
    fi
}

zz() {
    cd "$(_z -l 2>&1 | sed 's/^[0-9,.]* *//' | fzf -q "$_last_z_args")"
}

bindkey "${terminfo[khome]}" beginning-of-line
bindkey "${terminfo[kend]}" end-of-line

#gnome-shell-extension-tool -e run-or-raise@edvard.cz &> /dev/null

if [ $TILIX_ID  ] || [ $VTE_VERSION  ]; then
            source /etc/profile.d/vte.sh
fi

Thanks. I will find out why it does not work for me.

The copy & paste was a bit screwy - it duplicated some lines, have corrected.

I can’t understand why it’s not working for me, I’ve already tried everything :frowning_face:

https://bbs.archlinux.org/viewtopic.php?id=230609

h() { if [ -z "$*" ]; then history 1; else history 1 | egrep "$@"; fi; }

what the hell does that do!!!!!!

Yes it works, thanks! :beer:

Now I'm trying to configure PROMT for console tty.
For the terminal, powerlevel9k in the console does not look very good.

Do you have customized for TTY?

Tmux & spaceship prompt.

I was happy early, this feature:

h() { if [ -z "$*" ]; then history 1; else history 1 | egrep "$@"; fi; }

Does not remove duplicates! And just finds them!

My hack has one bug, if you execute the last command in history and close the terminal, then the command disappears from the history.
So that the command does not disappear, you need to either enter any other command or not use the parameters:

setopt HIST_IGNORE_ALL_DUPS
setopt HIST_IGNORE_DUPS

But then there will be one take of the last team:

zshc
aaaa
bbbb
cccc
aaaa

OR

zshc
bbbb
cccc
aaaa
aaaa

But I think with the parameter:

setopt HIST_FIND_NO_DUPS

It can be survived. :grinning:

In principle, this bug can be fixed, but I cannot understand how to intercept the event when zsh overwrites the history file. :worried: To launch a hack not on preexec, but on this event!

Changed the hack - hack No. 2. Assigned to Shift+Tab hotkey, you can assign to your own.

I’ve made the changes in order to get rid of the bug that I’ve talked about and not to overwrite the history file every time before executing the command in the terminal, this is not necessary.

Forum kindly sponsored by