2. BASIC SED
2.1. What is sed?
"sed" stands for Stream EDitor. Sed is a non-interactive editor, written by the late Lee E. McMahon in 1973 or 1974. A brief history of sed's origins may be found in an early history of the Unix tools, at <http://www.columbia.edu/~rh120/ch106.x09>.
Instead of altering a file interactively by moving the cursor on the screen (as with a word processor), the user sends a script of editing instructions to sed, plus the name of the file to edit (or the text to be edited may come as output from a pipe). In this sense, sed works like a filter -- deleting, inserting and changing characters, words, and lines of text. Its range of activity goes from small, simple changes to very complex ones.
Sed reads its input from stdin (Unix shorthand for "standard input," i.e., the console) or from files (or both), and sends the results to stdout ("standard output," normally the console or screen). Most people use sed first for its substitution features. Sed is often used as a find-and-replace tool.
sed 's/Glenn/Harold/g' oldfile >newfile
will replace every occurrence of "Glenn" with the word "Harold", wherever it occurs in the file. The "find" portion is a regular expression ("RE"), which can be a simple word or may contain special characters to allow greater flexibility (for example, to prevent "Glenn" from also matching "Glennon").
My very first use of sed was to add 8 spaces to the left side of a file, so when I printed it, the printing wouldn't begin at the absolute left edge of a piece of paper.
sed 's/^/ /' myfile >newfile # my first sed script sed 's/^/ /' myfile | lp # my next sed script
Then I learned that sed could display only one paragraph of a file, beginning at the phrase "and where it came" and ending at the phrase "for all people". My script looked like this:
sed -n '/and where it came/,/for all people/p' myfile
Sed's normal behavior is to print (i.e., display or show on screen) the entire file, including the parts that haven't been altered, unless you use the -n switch. The "-n" stands for "no output". This switch is almost always used in conjunction with a 'p' command somewhere, which says to print only the sections of the file that have been specified. The -n switch with the 'p' command allow for parts of a file to be printed (i.e., sent to the console).
Next, I found that sed could show me only (say) lines 12-18 of a file and not show me the rest. This was very handy when I needed to review only part of a long file and I didn't want to alter it.
# the 'p' stands for print
sed -n 12,18p myfile
Likewise, sed could show me everything else BUT those particular lines, without physically changing the file on the disk:
# the 'd' stands for delete
sed 12,18d myfile
Sed could also double-space my single-spaced file when it came time to print it:
sed G myfile >newfile
If you have many editing commands (for deleting, adding, substituting, etc.) which might take up several lines, those commands can be put into a separate file and all of the commands in the file applied to file being edited:
# 'script.sed' is the file of commands # 'myfile' is the file being changed
sed -f script.sed myfile # 'script.sed' is the file of commands
It is not our intention to convert this FAQ file into a full-blown sed tutorial (for good tutorials, see section 2.3). Rather, we hope this gives the complete novice a few ideas of how sed can be used.
2.2. What versions of sed are there, and where can I get them?
2.2.1. Free versions
Note: "Free" does not mean "public domain" nor does it necessarily mean you will never be charged for it. All versions of sed in this section except the CP/M versions are based on the GNU general public license and are "free software" by that standard (for details, see http://www.gnu.org/philosophy/free-sw.html). This means you can get the source code and develop it further.
At the URLs listed in this category, sed binaries or source code can be downloaded and used without fees or license payments.
184.108.40.206. Unix platforms
ssed is the version recommended by the FAQ maintainers, since it shares the same codebase with GNU sed, has the most options, and is free software (you can get the source). Though there were earlier version of ssed distributed, sites for these are not being listed.
GNU sed v4.0.5
This is the latest official version of GNU sed. It offers in-place text replacement as an option switch.
BSD multi-byte sed (Japanese)
Based on the latest version of GNU sed, which supports multi-byte characters.
GNU sed v3.02.80
An alpha test release which was the base for the development of ssed and GNU sed v4.0.
GNU sed v3.02a
Interim version with most features of GNU sed v3.02.80.
GNU sed v3.02
GNU sed v3.02-8
source code and binaries for Debian GNU/Linux
For some time, the GNU project <http://www.gnu.org> used Eric S. Raymond's version of sed (ESR sed v1.1), but eventually dropped it because it had too many built-in limits. In 1991 Howard Helman modified the GNU/ESR sed and produced a flexible version of sed v1.5 available at several sites (Helman's version permitted things like \<...\> to delimit word boundaries, \xHH to enter hex code and \n to indicate newlines in the replace string). This version did not catch on with the GNU project and their version of sed has moved in a similar but different direction.
sed v1.3, by Eric Steven Raymond (released 4 June 1998)
Eric Raymond <email@example.com> wrote one of the earliest versions of sed. On his website <http://www.catb.org/~esr/> which also distributes many freeware utilities he has written or worked on, he describes sed v1.1 this way:
"This is the fast, small sed originally distributed in the GNU toolkit and still distributed with Minix. The GNU people ditched it when they built their own sed around an enhanced regex package -- but it's still better for some uses (in particular, faster and less memory-intensive)." (Version 1.3 fixes an unidentified bug and adds the L command to hexdump the current pattern space.)
GNU sed v3.02.80
GNU sed v3.02
http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2-bin.zip # binaries http://hobbes.nmsu.edu/pub/os2/util/file/sed-3_02-r2.zip # source
220.127.116.11. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)
GNU sed v4.0.5
32-bit binaries and docs. Precompiled versions not available (yet).
GNU sed v3.02.80
32-bit binaries and docs, using DJGPP compiler. For details on new features, see Unix section, above.
http://www.student.northpark.edu/pemente/sed/sed3028a.zip # DOS binaries ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz # source ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028b.zip # binaries ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028d.zip # docs ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed3028s.zip # source
GNU sed v2.05
32-bit binaries, no docs. Requires 80386 DX (SX will not run) and must be run in a DOS window or in a full screen DOS session under Microsoft Windows. Will not run in MS-DOS mode (outside Win/Win95). We recommend using the latest version of GNU sed.
GNU sed v1.03
modified by Frank Whaley.
This version was part of the "Virtually UN*X" toolset, hosted by itribe.net; that website is now closed. Gsed v1.03 supported Win9x long filenames, as well as hex, decimal, binary, and octal character representations.
The Cygwin toolkit:
Formerly know as "GNU-Win32 tools." According to their home page, "The Cygwin tools are Win32 ports of the popular GNU development tools for Windows NT, 95 and 98. They function through the use of the Cygwin library which provides a UNIX-like API on top of the Win32 API." The version of sed used is GNU sed v3.02.
Minimalist GNU for Windows (MinGW):
According to their home page, "MinGW ('Minimalist GNU for Windows') refers to a set of runtime headers, used in building a compiler system based on the GNU GCC and binutils projects. It compiles and links code to be run on Win32 platforms ... MinGW uses Microsoft runtime libraries, distributed with the Windows operating system." The version of sed used is GNU sed v3.02.
sed v1.5 (a/k/a HHsed), by Howard Helman
Compiled with Mingw32 for 32-bit environments described above. This version should support Win95 long filenames.
sed v1.6 (from HHsed), by Walter Briscoe
This is a forthcoming version, now in beta testing, but with many new features. It corrects all the bugs in sed v1.5, and adds the best features of sedmod v1.0 (below). It is available in 16-bit and 32-bit compiled versions for MS-DOS. Sorry, no URLs available yet.
sed v1.5 (a/k/a HHsed), by Howard Helman
uncompiled source code (Turbo C)
DOS executable and documentation
sedmod v1.0, by Hern Chen
http://www.ptug.org/sed/SEDMOD10.ZIP http://www.student.northpark.edu/pemente/sed/sedmod10.zip ftp://garbo.uwasa.fi/pc/unix/sedmod10.zip
GNU sed v3.02.80
See section 18.104.22.168 ("Microsoft Windows"), above.
GNU sed v2.05
Does not run under MS-DOS.
GNU sed v1.18
32-bit binaries and source, using DJGPP compiler. Requires 80386 SX or better. Also requires 3 CWS*.EXE extenders on the path. See section 5.5 ("What is CSDPMI*B.ZIP and why do I need it?"), below. We recommend using a newer version of GNU sed.
http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip
GNU sed v1.06
16-bit binaries and source. Should run under any MS-DOS system.
ssed v2.2, by Chuck A. Forsberg
Written for CP/M, ssed (for "small/stupid stream editor) supports only the a(ppend), c(hange), d(elete) and i(nsert) options, and apparently doesn't support regular expressions. A -u switch will "unsqueeze" compressed files and was used mainly in conjunction with DIF.COM for source code maintenance. (file: ssed22.lbr)
change, by Michael M. Rubenstein
Rubenstein released a version of sed called CHANGE.COM (the TTOOLS.LBR archive member CHANGE.CZM is a "crunched" file). CHANGE.COM supports full RE's except grouping and backreferences, and its only function is global substitution. (file: ttools.lbr)
22.214.171.124. Macintosh v8 or v9
Since sed is a command-line utility, it is not customary to think of sed being used on a Mac. Nonetheless, the following instructions from Aurelio Jargas describe the process for running sed on MacOS version version 8 or 9.
(1) Download and install the Apple DiskCopy application
(2) Download and install Apple MPW
(3) Download and expand Matthias Neeracher's GNU sed for MPW. (They seem to have misnumbered the sed filename.)
(4) Enter the sed-3.02 directory and doubleclick the 'sed' file
(5) MPW Shell will open up. It will be a command window instead of a command line, but sed should work as expected. For example:
echo aa | sed 's/a/Z/g'<ENTER>
Note that ENTER is different from RETURN on an iMac. Apple also has its own version of sed on MPW, called "StreamEdit", with a syntax fairly similar to that of normal sed.
2.2.2. Shareware and Commercial versions
126.96.36.199. Unix platforms
[ Additional information needed. ]
A sizable set of Unix/C shell utilities designed for OS/2. Price is $350 in the US, $395 elsewhere, with FedEx shipping, unconditional guarantee, unlimited support and free updates. A demo version of the suite can be downloaded from this site, but a stand-alone copy of sed is not available.
188.8.131.52. Windows 95/98, Windows NT, Windows 2000
A sizable set of Unix/C shell utilities designed for Win9x, WinNT, and Win2K. Price is $350 in the US, $395 elsewhere, with FedEx shipping, unconditional guarantee, unlimited support and free updates. A demo version of the suite can be downloaded from this site, but a stand-alone copy of sed is not available.
Interix (formerly known as OpenNT) is advertised as "a complete UNIX system environment running natively on Microsoft Windows NT", and is licensed and supported by Softway Systems. It offers over 200 Unix utilities, and supports Unix shells, sockets, networking, and more. A single-user edition runs about $200. A free demo or evaluation copy will run for 31 days and then quit; to continue using it, you must purchase the commercial version.
MKS NuTCRACKER Professional
A different, yet related product line offered by MKS (Mortice Kern Systems, below); the awkward spelling "NuTCRACKER" is intentional. Various packages offer hundreds of Unix utilities for Win32 environments. Sed is not available as a separate product.
UnixDos is a suite of 82 Unix utilities ported over to the Windows environments. There are 16-bit versions for Win3.x and 32-bit versions for WinNT/Win95. It is distributed as uncrippled shareware for the first 30 days. After the test period, the utilities will not run and you must pay the registration fee of $50.
Their version of sed supports "\n" in the RHS of expressions, and increases the length of input lines to 10,000 characters. By special arrangement with the owners, persons who want a licensed version of sed only (without the other utilities) may pay a license fee of $10.
U/WIN is a suite of Unix utilities created for WinNT and Win95 systems. It is owned by AT&T, created by David Korn (author of the Unix korn shell), and is freely distributed only to educational institutions, AT&T employees, or certain researchers; all others must pay a fee after a 90-day evaluation period expires. U/WIN operates best with the NTFS (WinNT file system) but will run in degraded mode with the FAT file system and in further degraded mode under Win95. A minimal installation takes about 25 to 30 megs of disk space. Sed is not available as a separate file for download, but comes with the suite.
Mix C/Utilities Toolchest
According to their web page, "The C/Utilities Toolchest adds over 40 powerful UNIX utilities to your MS-DOS operating system. The result is an environment very similar to UNIX operating systems, yet 100% compatible with MS-DOS programs and commands." The toolchest costs $19.95, with source code available for an additional fee. Mix C's version of sed is not available separately.
MKS (Mortice Kern Systems) Toolkit
Sed comes bundled with the MKS Toolkit, which is distributed only as commercial software; it is not available separately.
Thompson Automation Software
The Thompson Toolkit contains over 100 familiar Unix utilities, including a version of the Unix Korn shell. It runs under MS-DOS, OS/2, Win3.x, Win9x, and WinNT. Sed is one of the utilities, though Thompson is better known for its version of awk for DOS, TAWK. The toolkit runs about $150; sed is not available separately.
2.3. Where can I learn to use sed?
Sed & Awk, 2d edition, by Dale Dougherty & Arnold Robbins (Sebastopol, Calif: O'Reilly and Associates, 1997) ISBN 1-56592-225-5 http://www.oreilly.com/catalog/sed2/noframes.html
About 40 percent of this book is devoted to sed, and maybe 50 percent is devoted to awk. The other 10 percent covers regexes and concepts common to both tools. If you prefer hard copy, this is definitely the best single place to learn to use sed, including its advanced features.
The first edition is also very useful. Several typos crept into the first printing of the first edition (though if you follow the tutorials closely, you'll recognize them right away). A list of errors from the first printing of sed & awk is available at <http://www.cs.colostate.edu/~dzubera/sedawk.txt>, and errors in the 2nd are at <http://www.cs.colostate.edu/~dzubera/sedawk2.txt>, though most of these were corrected in later printings. The second edition tells how POSIX standards have affected these tools and covers the popular GNU versions of sed and awk. Price is about (US) $30.00
Mastering Regular Expressions, 2d ed., by Jeffrey E. F. Friedl (Sebastopol, Calif: O'Reilly and Associates, 2002) ISBN 0-596-00289-0 http://regex.info http://www.oreilly.com/catalog/regex2/ http://public.yahoo.com/~jfriedl/regex/ (for the first edition)
Knowing how to use "regular expressions" is essential to effective use of most Unix tools. This book focuses on how regular expressions can be best implemented in utilities such as perl, vi, emacs, and awk, but also touches on sed as well. Friedl's home page (above) gives links to other sites which help students learn to master regular expressions. His site also gives a Perl script for determining a syntactically valid e-mail address, using regexes:
Awk und Sed, by Helmut Herold. (Bonn: Addison-Wesley, 1994; 288 pages) 2nd edition to be released in March 2003 ISBN 3-8273-2094-1 http://www.addison-wesley.de/main/main.asp?page=home/bookdetails&ProductID=37214
2.3.2. Mailing list
If you are interested in learning more about sed (its syntax, using regular expressions, etc.) you are welcome to subscribe to a sed-oriented mailing list. In fact, there are two mailing lists about sed: one in English named "sed-users", moderated by Sven Guckes; and one in Portuguese named "sed-BR" (for sed-Brazil), moderated by Aurelio Marinho Jargas. The average volume of mail for "sed-users" is about 35 messages a week; the average volume of mail for "sed-BR" is about 15 messages a week.
sed-BR mailing list: http://br.groups.yahoo.com/group/sed-br/ sed-users mailing list: http://groups.yahoo.com/group/sed-users/
To subscribe to sed-users, send a blank message to:
To unsubscribe from sed-users, send a blank message to:
2.3.3. Tutorials, electronic text
The original users manual for sed, by Lee E. McMahon, from the 7th edition UNIX Manual (1978), with the classic "Kubla Khan" example and tutorial, in formatted text format:
The source code to the preceding manual. Use "troff -ms sed" to print this file properly:
"Do It With Sed", by Carlos Duarte
"Sed: How to use sed, a special editor for modifying files automatically", by Bruce Barnett and General Electric Company
U-SEDIT2.ZIP, by Mike Arst (16 June 1990)
ftp://ftp.cs.umu.se/pub/pc/u-sedit2.zip ftp://ftp.uni-stuttgart.de/pub/systems/msdos/util/unixlike/u-sedit2.zip ftp://sunsite.icm.edu.pl/vol/wojsyl/garbo/pc/editor/u-sedit2.zip ftp://ftp.sogang.ac.kr/pub/msdos/garbo_pc/editor/u-sedit2.zip
U-SEDIT3.ZIP, by Mike Arst (24 Jan. 1992)
http://www.student.northpark.edu/pemente/sed/u-sedit3.zip CompuServe DTPFORUM, "PC DTP Utilities" library, file SEDDOC.ZIP
Another sed FAQ
sed-tutorial, by Felix von Leitner
"Manipulating text with sed," chapter 14 of the SCO OpenServer "Operating System Users Guide"
"Combining the Bourne-shell, sed and awk in the UNIX environment for language analysis," by Lothar Schmitt and Kiel Christianson. This basic tutorial on the Bourne shell, sed and awk downloads as a 71-page PostScript file (compressed to 290K with gzip). You may need to navigate down from the root to get the file.
ftp://ftp.u-aizu.ac.jp/u-aizu/doc/Tech-Report/1997/97-2-007.tar.gz available upon request from Lothar Schmitt <firstname.lastname@example.org>
2.3.4. General web and ftp sites
http://sed.sourceforge.net/grabbag # Collected scripts http://main.rtfiber.com.tw/~changyj/sed/ # Yao-Jen Chang http://www.math.fu-berlin.de/~guckes/sed/ # Sven Guckes http://www.math.fu-berlin.de/~leitner/sed/ # Felix von Leitner http://www.dbnet.ece.ntua.gr/~george/sed/ # Yiorgos Adamopoulos http://www.student.northpark.edu/pemente/sed/ # Eric Pement
http://spacsun.rice.edu/FAQ/sed.html ftp://algos.inesc.pt/pub/users/cdua/scripts.tar.gz (sed and shell scripts)
"Handy One-Liners For Sed", compiled by Eric Pement. A large list of 1-line sed commands which can be executed from the command line.
"Handy One-Liners For Sed", translated to Portuguese
The Single UNIX Specification, Version 3 (technical man page)
Getting started with sed
masm to gas converter
sample uses of sed in batch files and scripts (Benny Pederson)
dc.sed - the most complex and impressive sed script ever written. This sed script by Greg Ubben emulates the Unix dc (desk calculator), including base conversion, exponentiation, square roots, and much more.
If you should find other tutorials or scripts that should be added to this document, please forward the URLs to the FAQ maintainer.
Other Stuff On This Site
Emacs for Vi Programmers