Tuesday, December 4, 2012

Read through all values in an array in PostgreSQL

Same applies to Greenplum:

select col1,array1 from schema.table WHERE 'value_to_search_for'= ANY(array1);

array1 is a column in the table with type name[] in this case

Sunday, December 2, 2012

Flash bios Dell Latitude E4300 running Fedora 17

I did not figure this out on my own. I cobbled together information from a few sources. Run the first six steps from here (listed below). To get my new menu item to show up upon boot, so I could actually flash the bios, I started with 3.2 here. I simply added the menuentry {} section listed in step 2 below. I then ran grub2-makeconfig.
  1. Get the bios update from Dell (exe file)
  2. Get biosdisk from the Dell projects site[1]
  3. Get "memdisk" (Fedora: "syslinux" package)
  4. modify the biosdisk script, set the "baseDir" variable (first line after the comments) to the directory to which you extracted the biosdisk archive. The other option is to copy dosdisk.img to the default location ("/usr/share/biosdisk").
  5. Create a bootdisk as root: "./biosdisk mkimage <exe filename>"
  6. After the last step there should be a img file in /tmp (e.g. "E4300A24.img") which contains the bios flasher program. Move that file to /boot.
  7. As root, cp /boot/grub2/grub /boot/grub2/grub.ORIG
  8. Copy memdisk to /boot: cp /usr/share/syslinux/memdisk /boot
  9. Edit /etc/grub.d/40_custom add
    menuentry 'Dell Bios Update A24'{
     linux16 /memdisk
     initrd16 /E4300A24.img
    }
  10. grub2-makeconfig -f /boot/grub2/grub
Reboot. You should have a menu option 'Dell Bios Update A24'. Select that and press enter.

Time machine backups fail on new Toshiba drive

I did not develop this solution on my own. I found it in the Apple discussion forums, specifically this thread. Here are the details.

I purchased a new external drive, the Toshiba Canivo Basics 1.5TB model number  HDTB115XK3BA at Best Buy. I then follow the directions here to copy my old backup to the new drive. I went to back it up after the transfer and it immediately failed. The transfer had completed while I was at work so it had been 6+ hours since the transfer completed. That was plenty of time for the new hard drive to go to sleep. Turns out this is a habit of Toshiba drives from what I've been reading. I unchecked the option Put the hard disk(s) to sleep when possible in System Preferences/Engergy Saver and rebooted but that didn't fix the problem. I then came across a post about creating a crontab job to touch a file every minute to keep the drive away. Brilliant. So I now have the following crontab job that runs as the user I log in with every day.

* * * * * touch /Volumes/BackupDisk/test.txt

So far that seems to have done the trick.

Monday, July 9, 2012

Greenplum external table single column of data

Had to work with some data that was one column of data in a fixed width format. Solution was to create an external table that was one column then use substr function to pull out the data.

External table:

CREATE EXTERNAL TABLE trmtmp_ext_table (
    col1 text
)
LOCATION (
        'gpfdist://host-1:port/path/trmtmp.out',
        'gpfdist://host-2:port/path/trmtmp.out',
        'gpfdist://host-3:port/path/trmtmp.out'
)
FORMAT 'text'
LOG ERRORS INTO trmtmp_err_ext_table SEGMENT REJECT LIMIT 5 ROWS;

View of the external table:

DROP VIEW trmtmp_view_ext_tbl;
CREATE VIEW trmtmp_view_ext_tbl AS (
SELECT
    substr(col1,1,2) AS col1,
    substr(col1,3,5) AS col2
FROM  
    trmtmp_ext_table
);

Extrapolate from there.

Thursday, July 5, 2012

Backup script for Linux or OS X

A simple backup script for backing up my laptop. I have an OS X and Linux laptop so I wanted one script to put on both of them:


#!/usr/bin/env

homedir="$(echo $HOME)"
dom=$(date +%d)
###################
# START CONFIGURATION
###################

# On the 15th of the month run an rsync --delete to clean out old file. Otherwise just run a regular rsync.
if [ ${dom} -eq 15 ]; then
  rsynccmnd="rsync --delete -avhz --stats --progress"
else
  rsynccmnd="rsync -avhz --stats --progress"fi

# Path to the exclude file
host=$(uname -a|cut -d ' ' -f 1)
if [ "${host}" = "Darwin" ]; then
 # OS X
 excludefile="path to exclude file for OS X"
 #EXAMPLE
 # excludefile="${homedir}/Documents/Backup/backuplaptop_exclude_osx.txt"
else
 # Linux
 #excludefile="path to exclude file for Linux"

 #EXAMPLE
 # excludefile="${homedir}/Documents/Backup/backuplaptop_exclude_linux.txt"

fi

# Information for destination server
dstsrvr="destination.server"
dstusr="username on destination server" # if different from local uid

# Destination directory

#Linux version
#dstdir="Backups/$(hostname)/"

#OS X version
dstdir="Backups/$(hostname -s)/"
vmdstdir="Backups/vm/"

# Information for localhost

# Location of VMware directory on local machine
vmloc="path to virtual machines"
# EXMPLE
# vmloc="Documents/VirtualMachines/windows7vm.vmwarevm"
# DOW to backup VMware directory. Sunday is 0. Saturday is 6.
vmday=1
###################
# END CONFIGURATION
###################

start=$(date +%Y%m%d_%H%M)
echo "===================="
echo " Backup START: ${start}"
echo "===================="
echo " ====="
echo " Backup regular files"
echo " ====="
  ssh ${dstusr}@${dstsrvr} "if [ ! -d \"${dstdir}\" ]; then mkdir -p \"${dstdir}\"; fi"
  ${rsynccmnd} ${homedir}/ --exclude-from=${excludefile} ${dstusr}@${dstsrvr}:${dstdir}

dow=$(date +%w)
if [ ${dow} -eq ${vmday} ]; then
  echo " ====="
  echo " Backup VMware session: ${start} "
  echo " ====="
  ssh ${dstusr}@${dstsrvr} "if [ ! -d \"${vmdstdir}/${vmloc}\" ]; then mkdir -p \"${vmdstdir}/${vmloc}\"; fi"
  ${rsynccmnd} ${homedir}/${vmloc}/ ${dstusr}@${dstsrvr}:${vmdstdir}/${vmloc}/
fi
endtime=$(date +%Y%m%d_%H%M)
echo "===================="

echo " Backup START: ${start}"
echo " Backup END: ${endtime}"

echo "===================="

Friday, June 15, 2012

Data skew in Greenplum

Running this on a Greenplum DB 4.2 system will show you where you have data skew > 10%.

From the Greenplum admin guide gp_skew_idle_fractions:

select * from gp_toolkit.gp_skew_idle_fractions where siffraction > 0.1;

sifoid: objectid of the table
sifnamespace: schema name
sifrelname: table name
siffraction: The percentage of the system that is idle during a table scan, which is an indicator of uneven data distribution or query processing skew. For example, a value of 0.1 indicates 10% skew, a value of 0.5 indicates 50% skew, and so on. Tables that have more than 10% skew should have their distribution policies evaluated.


Friday, May 18, 2012

FTP and OS X

Just ran into something I haven't seen before. I was attempting to upload information to an ftp site. I opened Finder, entered ftp://sitename, and a connection was made and a new finder window opened. Tried to rsync in a terminal over to the /Volume/sitename but was told it was read-only. Same result when dragging and dropping in Finder.
Ran ftp sitename from a Terminal and it all worked. I was able to put files on the server.
Guess I need to lookup up and figure out what in the heck Finder does different than command line ftp.

UPDATE: Ok, after some searching I learned Finder mounts FTP as read-only. Don't know why but at least I now know.

Wednesday, May 2, 2012

Show running queries on Postgresql/Greenplum

I've got some queries that run for a long time. Sometimes longer than others. I wanted to see how long active queries have been running. Here's a view to consistently do that:

CREATE VIEW public.view_activequeries AS ( SELECT age(query_start, backend_start) AS queryage,* FROM pg_stat_activity WHERE current_query NOT LIKE '%IDLE%' ORDER BY queryage DESC);

Then you just run:

select * from public.view_activequeries;

You get a list of all active queries in descending order of how long they have been running, i.e. longest running queries listed first.

ssh-agent start upon login to RHEL 5 machine

---> All credit goes here <---

In case that ever disappears:

place in .bash_profile
####SSHagent settings####
SSH_ENV="$HOME/.ssh/environment"
function start_agent {
     echo "Initializing new SSH agent..."
     /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
     echo succeeded
     chmod 600 "${SSH_ENV}"
     . "${SSH_ENV}" > /dev/null
     /usr/bin/ssh-add;
}
# Source SSH settings, if applicable
if [ -f "${SSH_ENV}" ]; then
     . "${SSH_ENV}" > /dev/null
     ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
         start_agent;
     }
else
     start_agent;
fi

Monday, April 30, 2012

Sending emai in a perl script using sendmail

I needed a quick way to send an email when a script was done. Did some digging around and converted another CGI script to a plain perl script and finally got the syntax correct.
 #!/usr/bin/env perl
my $sendmail = "/usr/sbin/sendmail -t";
my $reply_to = "Reply-to: replyto\@addresshere\n";
my $subject = "Subject: Some amazing subject\n";
my $send_to = "To: destination\@addresshere\n";
open (SENDMAIL, "|$sendmail") or die "Cannot open sendmail: $!\n";
print SENDMAIL $reply_to;
print SENDMAIL $subject;
print SENDMAIL $send_to;
print SENDMAIL "Content-type: text/plain\n";
print SENDMAIL "Content here. You could also put the content in a variable above.";
close(SENDMAIL);

Wednesday, April 18, 2012

Active Directory account caching on OS X Lion

Looks like account caching of AD accounts is dependent on the same Kerberos preauthentication issue I blogged about a couple of posts back. So unchecking the box "Do not require Kerberos preauthentication" on our user accounts resulted in network credentials being cached and network accounts able to log in even when the notification said network accounts were unavailable.

Monday, April 16, 2012

.ssh permissions

I've had the same ssh keys for years. I just rsync them to a new system when I get one. I always seem to end up with mucked up permissions moving them around and never seem to remember how the permissions were set. Here's my reminder.

~/.ssh 700
~/.ssh/authorized_keys 600
~/.ssh/config 644
~/.ssh/id_dsa 600
~/.ssh/id_dsa.pub 644
~/.ssh/known_hosts 644

Friday, April 13, 2012

Unable to join Mac Lion 10.7.3 client to an active directory domain


Kept receiving the error using the GUI, "the plugin has encountered an error processing request". On the command line I'd get that plus the 10001 error code. My domain admin account would not work but a coworkers would. The difference? My account had the box checked "Do not require Kerberos preauthentication". As soon as that was unchecked, I was able to use my domain admin account to bind a 10.7.3 machine to an Active Directory Domain. Note that my admin account had worked on binding 10.x machines BEFORE 10.7 just fine with that box checked. We had that box checked because we have Linux machines that authenticate off our active directory domain and for accounts to authenticate, they needed that box checked.

Friday, March 23, 2012

Right side of OS X menu bar frozen, take two

So there's a command line solution to a frozen right side menu bar in OS X that I posted about a while back. You can also fix it using a gui if you are so inclined.

1. Open Activity Monitor from /Applications/Utilities
2. Find process name SystemUIServer
3. Click Quit then Force Quit

Right side of menu bar should now restart

Saturday, March 10, 2012

Command line mailing in Linux

Mail a file with a subject and attach a file.

mail -s "Subject goes here" 'email addresses here, comma separated if more than one' < /path/to/filetoattachment

Thursday, March 8, 2012

/var/audit filling with files on OS X

Coworker discovered that installed antivirus software enabled auditing, for real time auto-protect, which results in /var/audit filling up with files. Details about the configuration file and related commands can be found here:

https://developer.apple.com/library/mac/#documentation/Darwin/Reference/Manpages/man5/audit_control.5.html

Saturday, February 18, 2012

Restart VMware Fusion from the command line

Via : http://communities.vmware.com/thread/195941

sudo /Library/Application\ Support/VMware\ Fusion/boot.sh --restart

Monday, February 13, 2012

Error 15 when starting Adobe Creative Suite products

Recently had a problem where a user would install the CS suite(same thing happened with v4 and v5) but when they would start any of the products an Error 15 would be the result and the suggestions was to reinstall the product. It finally dawned on me that an aggressive security setting had been setup according to this document:

http://support.apple.com/kb/ht2202

We had set ours to 077 and it was preventing CS products from running. As soon as I removed that setting from launchd-user.conf everything worked perfectly.

Tuesday, January 24, 2012

Screenshots on the Mac

I keep forgetting the key combinations. Now I have them forever marked.

How to get screenshots on the Mac via MacRumors

Wednesday, January 18, 2012

Remote Management OS X command line

Via: http://support.apple.com/kb/HT2370#

To enable Remote Managment:

/System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate

To disable Remote Management:

/System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart - deactivate

Friday, January 6, 2012

Alter resource queue in Greenplum

The documentation is a little off. To alter a resource queue in Greenplum run the following:

ALTER RESOURCE QUEUE queue_name COST THRESHOLD your_new_threshold;

The documentation states that you use WITH:

ALTER RESOURCE QUEUE myqueue WITH (MAX_COST=3e+10);

That, however, never worked for me. If you run \h ALTER RESOURCE QUEUE from a psql prompt, it gives you the proper syntax.