UNIX ENTERPRISE CONSULTANT

PROVEN SYSTEMS EXPERTISE IN A CAREER SPANNING OVER 19 YEARS OF TECHNICAL SYSTEM MANAGEMENT EXPERIENCE IN BUSINESS CRITICAL ENVIRONMENTS  

CORE COMPETENCIES

EDUCATION, CERTIFICATION & TRAINING

·        Supporting almost 40 HPUX servers in a clustered development and production environment running Oracle, Lawson, etc. and utilizing EMC Symmetrix and Clariion disk drive systems.

·        PROJECT PLANNING and IMPLEMENTATION:  Researched the impact of new mandated Daylight Savings Time changes and required upgrades for a variety of operating system platforms including HPUX, AIX, Sun Solaris, Red Hat Linux, Windows, VMS, etc. and impact on applications and system services such as Java and applications that use Java.  Prepared and implemented project plan for HPUX servers.  Upgrading multiple versions of Java on HPUX servers.  Found and brought to the attention of HP senior and executive management numerous issues with HPUX with regard to stability for daylight saving time.  Made various recommendations including the creation of HP’s daylight saving time website.

·        SYSTEM ADMINISTRATION:  Conducted audit of HPUX servers to ensure that there was no boot-related single points of failure and made corrections to ensure proper booting in case of a disk failure.  Standardized the process of making automated emergency boot images of each server with notification.  Implementing a solution for monitoring by HP Openview and Remedy ticket generation.  Assisted with the diagnosis and resolution of several problems on Sun Solaris servers.  Working on tear-down of HPUX Service Guard cluster and proper removal of EMC drives.  Support business lines with upgrades of products, etc. to include diagnosing and finding solutions for various problems.

·        NETBACKUP:  Assist Netbackup administrator with file restores, backups, etc. in a SAN-based Netbackup environment with multiple media servers, etc.

·        Facilitated the installation and configuration of new healthcare analysis software (EBMConnect) on AIX 5.3 servers.  Project involves the extraction of data from the mainframe to AIX servers for processing and transfer back to the mainframe and to Windows SQL servers. 

·        SYSTEM ADMINISTRATION:  Designed and configured multiple custom environments for business lines to use and upgrade EBMConnect with appropriate analysis and job (Tivoli Workflow Scheduler) processing with interfering with the use of EBMConnect.  Designed and implemented process for source control for over 7,000 lines of code in scripts for each environment.

·        SCRIPTING:  Designed and menu-based system to simplify software support and mitigate the frustration of working with the UNIX command line interface.  Wrote Perl scripts to do binary searches on large data files for ad-hoc queries.  Wrote scripts for Tivoli Workflow Scheduler job stream to validate input data, run analysis software and archive results, etc.  Primary scripts spawn off scripts to validate sections of large files in parallel to save time, monitor the progress of the analysis software and ensure that the process does not exceed a time limit.  Script uses Korn shell, awk and Perl.  Scripts support parallel processing of regions (40+ hours runtime, each).  Performance and resource data collection and automatic sorting and archiving of data with automatic roll-off.  Scripts send automated emails with status and attached reports.  Users can set option in a configuration file.

·        CAPACITY PLANNING:  Evaluated disk requirements and instituted processes to minimize disk requirements resulting in cost avoidance of several hundred GB of disk space.

·        UNIX SUPPORT:  Qualified suitability of software for AIX 5.3 and JFS2, found problems with data that causes the software to fail and wrote UNIX scripts that validates data prior to analysis by software. Solved problems and provided recommendations for application and project teams.

·        PROJECT PLANNING:  Prepared numerous project plans for EBMConnect analysis software upgrade and qualification.

·        HP-UX ADMINISTRATION, HP SUPERDOME, PATCH AND KERNEL MANAGEMENT:  Support of L-Class to Superdome Class PA-RISC servers, rx2420 Itanium Servers and Linux servers in a fast-paced FDA validated production environment.  Solved system problems, software installation, etc.  Evaluated Kernel parameter settings for IA64 and PA-RISC servers.  Teamed with SAN administrator (HP xp1024 SAN environment) to add disk volumes, etc.    Prepared and implemented change controls.  Worked on Oracle upgrade project for HP Superdome. Investigated HP Superdome performance problems and made kernel changes and patch installations. 

·        SCRIPTING, DEVELOPMENT:  Designed and developed Korn shell scripts to process Oracle data mart extractions, etc. in a Tivoli environment.  Scripts called SQLPLUS and stored procedures, checked for resources, trapped errors, etc.  Created job definition files to minimize the number of scripts and implemented version control to simplify support.  Wrote detailed documentation for each script and the overall process.

·        TRAINING:  Developed curriculum and presented classes on UNIX shell, VI editor, Korn shell scripting with awk, sed and version control in a UNIX environment.

·        HP-UX 11i, MC SERVICE GUARD, SECURITY, SARBANES-OXLEY:  RP8400, RP7400, N and L-series servers.  Created clusters and packages in an Oracle environment.  Programmed script that checks security compliance with Sarbanes-Oxley – eliminating the bulk of a costly manual verification process.  Implemented corrective action for security deviations.  Analyzed kernel parameters, provided system administration support, patched servers, created and extended logical volumes, etc.  Used NESSUS to check for security holes on HP-UX servers.  Configured Symark PowerBroker on several servers with remote logging and control.  Built new server and migrated EMC Symmetrix disks.  Upgraded and modified configurations of SAMBA and Apache.  Used Remedy to track project tasks and support calls.

·        HP-UX 11i and 11.0, EMC CONTROL CENTER, SYMMETRIX, HP SERVICE GUARD, HP CONTINENTAL CLUSTER:  Supported RP8500 series, RP7410 series, K-series and D330s in a HP MC Service-Guard cluster and HP Continental wide area cluster environment running Oracle databases, Filenet and custom applications.  Created BCVs, LUN masking, etc. using EMC Control Center.  Defined R1+R2 pairs and data groups, splitting and restoring R1+R2 links, etc. Worked with team to install new storage area network with HP servers, Novell, etc.  Prepared servers for EMC Centera by patching servers, updating kernel parameters, etc.

·        MC SERVICE GUARD:  Created and updated packages in multiple clusters.  Defined new node and added to cluster, etc.  Updated configuration to Continental wide-area cluster.  Managed nodes and clusters to set up new applications (Oracle Financials, etc.)

·        HP OPENVIEW:  Installed and configured HP Openview Network Node Manager.

·        DNS, BUSINESS CONTINUANCE:  Salvaged DNS services by rebuilding it and writing scripts to simplify DNS administration for the Workers Compensation Commission.  Wrote scripts to rebuild databases, update configuration files and check name service integrity on master and slave servers.  Upgraded Bind 8.x to 9.x.  Developed DNS disaster recovery plan and implemented automated replication to recovery site.

·        OMNIBACK/DATA PROTECTOR:  Updated HP Omniback/Data Protector to 5.1.  Set up reports and Awk scripts to automatically post backup reports to the administration website.

·        WEB SERVER DESIGN AND MANAGEMENT, BUSINESS CONTINUANCE:  Configured website for system reports, automated configuration documentation, and ad-hoc notes for system and Filenet administrators using Korn Shell and Awk.  Set up automated replication system using UNIX tools and Awk to replicate the website to the recovery site on a schedule to ensure availability of critical recovery information. 

·        LINUX:  Built DNS slave server on SuSE S/390 (mainframe) LINUX.  Project involved getting source code, compiling and installing Bind, etc., reconfigured network interfaces and managing disk space using LVM (logical volume manager).

·        PROJECT MANAGEMENT/PLANNING:  Developed plan for the update of AppWorx system scheduler, including appropriate backups, web configuration, impact on MC Service Guard packages, client systems; checking technical details for configuration for business continuity, etc.  Providing UNIX technical support.

·        ADMINISTRATION:  Support and administration of HP-UX servers for production and development Filenet servers and Oracle.  Modified Sendmail configuration etc. to adjust production mail system.  Used TCP/IP network packet tracing, tcpdump, etc. to solve communication problems between Filenet servers.

 ·        SUN SOLARIS 8:  Evaluated Sun Solaris server for proper sizing for production software, finding it inadequate.  Subsequent testing confirmed conclusion in spite of vendor claims to the contrary.

·        SECURITY: Changed snmp configuration on servers, installed and configured tracking software, made various security recommendations.

·        HP-UX 11i, 11.0, 10.20, ADMINISTRATION, PROJECT MANAGEMENT/PLANNING:  Day-to-day support and special projects in a mixed HP-UX server environment ranging from D-class, K-class and V-Class servers in a Service Guard cluster running BAAN, BMC Scheduler (Control M), HP Openview and HP Omniback in addition to Oracle databases in a EMC SAN environment.  Configured Domain Name Service (DNS).  Prepared and implemented detailed command-by-command plan for the migration of NIS for over 1,500 accounts from one production server to another, including testing, roll-back plan, etc.  Upgraded Ignite on several servers.  Researched, developed and executed plan for the upgrade of a problematic software package that was causing problems and delaying invoicing, etc.  Coordinated technical details and licensing, etc. with software vendor, verify proper media, versions, patch levels, etc.

·        BUSINESS CONTINUANCE, EMC SAN:  Set up management server and wrote script that automatically captures full configuration of servers including disk and volume layout, kernel configuration, installed software, EMC Symmetrix disks (BCVs, etc.), MC Service Guard configuration, NIS configuration, lan and lan card configuration, name resolution configuration, backup software configuration, devices, key configuration files and memory configuration.  Fully automated to provide daily updates in html and text format.  Documentation supplements HP Ignite recovery tapes in case of failure.

·        HP OPENVIEW, HP OMNIBACK/DATA PROTECTOR, PROJECT MANAGEMENT/PLANNING:  Developed and implemented detailed plan for Omniback to Data Protector migration, requiring steps for operating system patches, database backup and conversion, back out plan, HP Openview upgrade, licensing and testing.  Provided ongoing Omniback support.  Conducted testing for backing up Windows 2000 and NT servers using HP Data Protector.  Developed script that is integrated with HP Openview to monitor NT web server.  

·        EMC SAN/NAS, HP-UX 11i, 11.0, ADMINISTRATION, ENTERPRISE MANAGEMENT AND MONITORING, ORACLE, PROJECT MANAGEMENT/PLANNING, TECHNICAL LEAD/TRAINING: UNIX system administration support of HP Superdomes, K, L, N and V-class servers in a large production environment that utilizes EMC Symmetrix SAN, NAS servers, EMC EDM backup software, Autosys, Veritas Netbackup, Connect Direct.  Most servers are Oracle production database servers.  Support included daily administration, trouble-shooting problems (TCP/IP, performance, NFS and automounter, disks and devices, kernel parameters, EDM backup, Omniback backup, etc.), support of Oracle, performance, etc. building of new systems, large data moves and special projects.  Configured Nike Array (EMC Clariion) disk subsystems.  Used EMC EDM and Veritas Netbackup to back up and restore files to HP servers.   Developed and executed numerous detailed project plans (command-by-command) with back-outs and timelines that required coordination with other technical and business-line support teams. Served as a de-facto technical lead, helping other team members with production problems and training them. 

·        BUSINESS CONTINUANCE, TRAINING: Built and tested a HP Ignite server to improve system recovery by automatically collecting and maintaining boot images across the network; reducing the probability of requiring a system to be rebuilt from scratch.  The “Ignite” server was adopted as the standard for operating system recovery for HP UNIX servers. 

·        BUSINESS CONTINUANCE, EMC, SCRIPTING: Engineered fully automated script that collects detailed as-built configuration information, including volume group information, disk configuration, devices, kernel parameters, installed software, LAN configuration details, installed patches, critical boot information, etc. and produces online html and text reports to enable HP-UX servers to be rebuilt to current configuration specification.  Script provides information in case of an Ignite image or backup failure.

·        EMC SAN/NAS, HP, PROJECT MANAGEMENT/PLANNING, ORACLE: Built new servers, developed and executed project plan to migrate Oracle database data via EMC BCVs and HP logical volumes (Veritas VxVM equivalent), maintaining data integrity and improving performance by striping disks, etc. from SCSI attached EMC Symmetrix drives to EMC SAN-based Symmetrix drives.  The executed plan was coordinated with other technical groups; improved performance by reviewing and modifying kernel parameters and file system parameters; estimated timelines for work completion, notification of other teams, back-out plan, etc.  Engineered synchronization of operating system data between two servers on an automated schedule to ensure integrity.  Migrated production data in HP Service Guard and Filenet cluster, etc.

·        WEB SERVER, SCRIPTING, BUSINESS CONTINUANCE: Designed and developed web site for UNIX system administrators to maintain web-based notes, reference documentation and job stream information for all jobs running on the servers.  Scripts build pages as they are navigated.  Overnight security reports, Autosys job definitions and system configuration reports are updated automatically.  The website simplified on-call support for system administrators by translating cryptic job names from operations to the necessary scripts, log and error files and job dependencies required to solve production problems.  Built redundant (disaster recovery) web server that is automatically synchronized with the primary web server with up-to-date server information.  Wrote scripts that captured the current value of kernel parameters for system monitoring. 

·        CONNECT DIRECT: Diagnosed and repaired data transfer problems between servers.

·        AUTOSYS:  Customized scripts to run under Autosys scheduler.  Diagnosed problems with production Autosys configuration, etc.  Migrated Autosys between servers.  Wrote scripts that used Autosys logs to produce weekly failure reports, etc.

·        SUN SOLARIS, SECURITY, SCRIPTING: Constructed daily automated password and security reports that feed data directly back to the server to produce web-based reports security reports for HP-UX and Sun Solaris servers.

·        HP SUPERDOME, EMC SAN/NAS, PROJECT MANAGEMENT/PLANNING, TECHNICAL TEAM LEAD:  Evaluated, with a team member, the architecture, performance potential, proper configuration and cost of systems, etc. of HP V2600 versus the HP rp8400 and HP Superdome for presentation to management.  Worked on configuration of 2 Superdome servers with 2 partitions each including kernel parameters, disk and logical volume layouts, disk striping, JFS file system (Veritas VxFS) tuning, software installation and configuration, etc.  Prepared substantial ports of the system build and software/data migration plans using EMC BCVs, UNIX copies, etc. in SAN environment.  Helped the project manager develop overall plan, defining options, technical issues, timing, etc.  One of two HP-UX consultants that lead the UNIX team for configuration, build and migration to the Superdome.  Development and production servers were delivered ahead of schedule. 

·        HP DATA PROTECTOR (OMNIBACK):  Updated data lists, diagnosed problems and managed backups and restores.  

·        SUN SOLARIS, BUSINESS CONTINUANCE, PROJECT MANAGEMENT/PLANNING: Developed technical disaster recovery plan for Sun Solaris CDARS server that runs multiple web servers that interact with other servers and systems (VAX and UNIX servers running Oracle Clinical, etc.).  Plan development included determining system interactions with other servers, print systems, network information system, etc., testing and verification, development of operating options to minimize dependencies.  Plan development was coordinated with teams in the United States, United Kingdom, India and Japan.  The plan was to be implemented after the core system was rebuilt per a standardized procedure.  Reviewed candidate patches for installation.

·        SUN SOLARIS, WEB SERVER, SCRIPTING: Documentation of the interaction of server software wherein scripts generate web pages, make database system calls, compress pages prior to sending to client, sends reports to document management systems on other servers, converts reports to different formats, etc.  Multiple web servers where there is complex interaction between the web servers, Perl, Java Script, SAS and SQL, Adobe Distiller, UNIX utilities, shell scripts, etc.  Reverse-engineered large and complex scripts in shell and Perl to determine web configurations, flow of software product use and ‘virtual’ Sun server configuration.

·        PERFORMANCE, HP-UX: Performance tuning and testing support on HP 9000 servers at remote facility. 

·        SUN SOLARIS, PERFORMANCE, SCRIPTING, ORACLE: Wrote scripts to capture performance metrics, etc. on Solaris test server.  Wrote scripts to collect Sun server hardware and operating system configuration.  Used scripting to determine the interaction of other servers (UNIX, VMS, NT, etc.) with primary production server over wide area network (India, United Kingdom, multiple US locations and Japan).  Wrote scripts to support day-to-day maintenance of Sun servers.  Supported quality assurance ORACLE DBA teams.

·        BUSINESS CRITICAL, HP-UX 11.0, 10.20, ORACLE: Upgraded system availability for all HP9000 K-series servers to 99.9%+ through improved systems management, configuration and monitoring in an Oracle database and data-mart environment with remote data feeds.  With DBA team, implemented standardized Oracle startup and shutdown across all servers.  Set up monitoring so that the almost all hardware problems are identified and corrected before a critical stage was reached without system downtime.

·        BUSINESS CONTINUANCE, SCRIPTING: Wrote automated disaster recovery program that collects configuration data, backup software and software depots for all HP-UX servers.  Configuration data includes all logical volume and disk information, kernel parameters, startup scripts, etc. for quick restoration and reconfiguration back to original system specifications.  A bootable tape is automatically generated to expedite the recovery process.

·        ENTERPRISE-WIDE MANAGEMENT & MONITORING, ORACLE: Modified and implemented a web-based monitoring tool to alert support teams should connectivity, key services or processes on UNIX or NT servers fail or disk space drops below thresholds.  Connectivity is monitored for Novell servers, routers and data feeds. 

·        SECURITY: Identified and implemented new security measures to reduce the probability of unauthorized access.  Only team in corporation to trap intrusion and notify corporate security.

·        PERFORMANCE, HP-UX 11.0, ORACLE: Reconfigured HP Measureware (performance data collection) and set up automated reports by application group to ease the evaluation of system performance.  Teamed with lead Oracle DBA to review and modify kernel parameters, evaluate snapshot file systems for backup, disk striping, file system parameters, etc.  Modified memory allocation to improve Oracle SGA performance.

·        WEB SERVER: Installed and configured web server software to provide system, as-built documentation and operations data for night-shift operators via web browsers.  Wrote scripts that generate html documents with links to log files, etc. that are automatically updated.  Web servers interact with web servers on other systems to eliminate unnecessary data redundancy.  Wrote scripts that can be activated via a browser to provide systems information and operations status.

·        BUSINESS CONTINUANCE, WEB SERVER: Built automated web-based system information collection for disaster recovery.

·        OS UPGRADES, PROJECT MANAGEMENT/PLANNING, ORACLE: Prepared and implemented plan to upgrade HP-UX 10.20 to 11.00, permitting system booting to either HP-UX 10.20 or 11.0 to ensure smooth transition during upgrade.  Developed coordinated plan with the DBA team for the transition of Oracle 7 to 8.

·        SUN SOLARIS, WEB SERVER, SCRIPTING: Installed and configured Solaris and Apache web server.  Installed compilers, etc.  Diagnosed communication problems for data feeds, resource management, etc.  Set up performance and monitoring agent to automatically send alerts.  

·        OS UPGRADES, HP-UX 9.x, 10.20, PROJECT MANAGEMENT/PLANNING: Primarily responsible for upgrading HP 9000 servers to 10.20 from 10.01 and 9.04 and updating the operating system for Y2K compliance.  Prepared and upgraded systems in a high-availability and manufacturing environment (HP Switchover).  Pinpointed and corrected numerous hardware and software related problems while preparing the systems for upgrade.  Developed and executed detailed upgrade plan for each site, incorporating their technical and business requirements to ensure full system functionality.  Reviewed Y2K upgrade plans and identified issues for independent testing including kernel changes and installation of EMC Clarion (Nike-Array) disk systems. 

·        ADMINISTRATION, ORACLE: Provided support for Legato, Tivoli and LAN Manager.  Provided support to Oracle DBAs.   Used Remedy for support call tracking.  Established HP software depots for Y2K patches and HP software for controlled installation.

·        EMC, HP-UX SERVICE GUARD: Provided production support in a large central data center with EMC disk subsystems, using HP’s Online JFS (Veritas VxFS). Made hardware and operating system changes in an HP Service Guard environment.

·        SYSTEM MANAGEMENT: 10 HP-UX K & T series servers running a data-mart with supporting database and development servers.  Standardized startup and shutdown procedures for all systems.  Simplified administration and improved system reliability.  Set up management server for patch and software depots and standardized the patch and software upgrade process.  Set up a private network to offload backup network and support traffic from the primary network.  Completely restructured the server backup process with Omniback (Data Protector), replacing a dysfunctional and unreliable backup process with organized backup.

·        BUSINESS CONTINUANCE, SCRIPTING: Engineered methodology and scripts for HP9000 disaster recovery program, including additional backup of special files, identification of configuration files and their automatic backup to other servers and tape.  Scripts automatically update data daily.  Wrote recovery procedure with checkpoints to ensure a reliable recovery.  Conducted recovery testing with Oracle DBA and middleware team at HP’s Performance Center and Sungard’s recovery center.  Recovered production server to full functionality.  Wrote scripts for automatic paging for system checks.

·        PLANNING, ADMINISTRATION: Lead system administrator for the move of servers to new facility.  Tasks included bringing systems up to patch level, installing CA-Unicenter Scheduler, configuring systems for DNS, preparing systems for move, developing and implementing a detailed system checkout, ensuring smooth system startup, production-level functionality and connectivity with the wide-area network.  Checked out systems at new facility to ensure proper functionality.

·        SECURITY: Set up internal web server for security group that provided self-maintaining security data and logs.  Researched, identified and instituted corrected numerous security holes including the modification of internet daemon services (including denial of services to selected clients).  Set up remote logging to other systems.

·        SOFTWARE SUPPORT: Supported Autoxfer software distribution for automatic distribution of data-mart reports to a sales force of 300.  Diagnosed and fixed problems.  Wrote scripts to facilitate smooth operation.  Trained a support team.

·        SUN SOLARIS:   Diagnosed and repaired problems on Sun print servers.

·        HP-UX, ADMINISTRATION, ORACLE, Enterprise Management & Monitoring: HP-UX consultant for development and production servers in an oracle database environment.

·        PERFORMANCE, KERNEL, ORACLE: Part of team to determine optimum UNIX kernel parameters in an Oracle database environment. Researched UNIX bottlenecks using various data and software, including HP Glance, HP PerfRx, vmstat, iostat, sar, etc.  Found and fixed critical bottleneck.

·        ADMINISTRATION: Researched and recommended a script-based backup plan for a 24 x 7 UNIX system Oracle environment.    Diagnosed problems and initiated corrective action, as required.  Performed complete installation and configuration of HP-UX (9.04).

·        SCRIPTING, ORACLE: Provided support for HP-UX development servers.  Developed scripts to monitor Oracle database instances and other system functions with automatic notification.  Wrote Korn shell scripts (over 1,900 lines of code) with embedded SQL and Hewlett-Packard Omniback software to perform hot backups of Oracle.  Scripts featured automatic delayed notification via pager of failure of critical functions and detailed logging of events.

·        ADMINISTRATION, HP-UX: Consulting services for Hewlett-Packard 9000 800 (T500) during conversion of IBM mainframe system to UNIX system.  Trained UNIX system administrator.

·        SCRIPTING: Developed Korn shell scripts to implement IBM mainframe GDG functionality, automatically sequencing data sets and automatic removal of old sets.  Developed menu-based Korn shell script to manage the migration of source code, object code, “JCL”, etc. from test to quality assurance and production.  Script contained extensive error checking and automatic archiving and version control.  Repaired and expanded Korn shell script that emulates IBM Job Control Language (JCL).

·        ADMINISTRATION, ENTERPRISE MANAGEMENT AND MONITORING: Day-to-day administration of 100-plus HP9000 UNIX-based workstations and Xterminals.  Supported engineering and drafting software - Unigraphics, Ansys, Interleaf, etc.  Resolved problems with NFS among multiple workstations.

·        PROJECT MANAGEMENT/PLANNING: Worked to bring all workstations under NIS control for ISO 9000 compliance.  Determined issues and resolutions, coordinated with networking, engineering, etc.

·