荔园在线
荔园之美，在春之萌芽，在夏之绽放，在秋之收获，在冬之沉淀
[回到开始]
[上一篇][下一篇]
发信人: jjksam (I want to believe...), 信区: Linux
标 题: 一篇优秀的BSD评论文章：作者是写VM的人(转寄)
发信站: 荔园晨风BBS站 (Thu Oct 11 11:40:22 2001), 转信
发信人: surbeyond (斜风细雨), 信区: FreeBSD
标 题: 一篇优秀的BSD评论文章：作者是写VM的人
发信站: BBS 水木清华站 (Tue Oct 9 18:00:27 2001)
Matt Dillon, not the famous actor but the kernel/VM FreeBSD hacker also
well
known for writting the Dice C compiler for the Amiga, is here with us today
for
an in-depth interview about everything regarding FreeBSD 50. This is the OS
that
all the techie
people are waiting for and presenting it as the most advanced,
technically-speak
ing, free OS of today. Additionally, we also include two mini interviews
with The
o de Raadt, the OpenBSD founder, and Jun-ichiro "itojun" Hagino from the
NetBSD C
ore Team.
Matt Dillon, VM and kernel FreeBSD team
1. What goodies BSD 5.0 is going to bring us?
Matt Dillon: There are at least a dozen projects going on in parallel and
we hav
e ripped up and replaced a great deal of code all over the kernel. So
much so tha
t we recently decided to extend the 5.0 release date another year to give
the man
y projects
a chance to get out of development mode and into stabilization mode. Due to
this
a good chunk of -current's features are also being slowly MFC'd into
-stable. No
t KSEs or SMPng though. I think Julian's comment in his KSE commit
message was so
mething on
the order of "X-MFC after: ha ha ha ha" :-)
KSEs and SMPng are the most visible projects (SMPng is being spear-headed
by Joh
n Baldwin), but we have also done a great deal of work on the network
stack, chec
ksum offloading, network drivers (especially GigE drivers), devfs (which is
now t
he default
in -current), crypto-quality random number generation, scaleability, and
new mac
hine ports. We are actively porting to IA64, PowerPC, and Sparc64, and even
thoug
h the Alpha is dying away our already-operational (in -stable) alpha port
is bein
g actively
developed to ensure that our code remains 64 bit clean and to provide
ground-bre
aking work for the other ports.
A good chunk of the work has already been MFCd to stable. For example,
-stable c
an push 900+ MBits (120 MBytes/sec 'netstat -in 1') over a TCP connection
on a DE
LL2550 with GigE using *normal* sized frames (mtu 1500). That's full
saturation.
A great deal of filesystem work has also been completed. Filesystem
snapshot sup
port (for UFS+Softupdates) is progressing nicely and may prove to be one of
the m
ost important new filesystem features in the 5.x system once it stabilizes.
There
are also
two new native features in UFS that have stabilized and have in fact been
MFCd t
o -stable: dirpref and dirhash. dirpref comes directly from Grigoriy
Orlov of Ope
nBSD and reworks the way directories are layed-out on disk, resulting in
huge dir
ectory and
file stat/open/create/remove performance gains. Some filesystem
operations have
improved by over 60x (6000%), and many of the common ones have improved
over 400%
. dirhash is a very low-overhead in-kernel whole-directory hashing
mechanism that
radically
improves the performance of directory operations.
There is much more. Some things, like dirhash and dirpref, can be easily
MFC'd t
o -stable. Others, such as softupdate's snapshot support, probably won't be
due t
o major dependancies on other current-only projects.
2. Which is the one feature that you would like most to add to the BSD
kernel?
Matt Dillon: I would like to see a native process and device-level
descriptor mi
gration capability. This isn't a new idea (few ideas in computer
architecture can
ever be called 'new'), but it is an idea whos time has come. I would
like to see
an
ability to migrate processes as well as their device-level state and couple
that
with external rerouting. So an I/O descriptor representing a TCP
connection coul
d be migrated entirely off the original machine, for example.
Process migration is a good basis to support Q.O.S. and maintainance issues
on t
odays platforms. As computing hardware becomes more powerful and we run
more serv
ices (and more connections, and more users) on any given box, the ability
to migr
ate
everything off a box in order to take it down for maintainance without
users not
icing that you are doing it has become the ultimate IT grail. The reason is
simpl
e: if you use a modern machine to its fullest potential and the system
crashes, y
ou are
potentially interrupting thousands of users rather then simply dozens.
The conce
pt of the 'maintainance-window' introduces the same problem, even with load
balan
cing and connection management and distribution technologies. The
'maintainance-w
indow'
concept is rapidly becoming unacceptable in today's full-on world.
In short, process migration would allow the open-source community to
begin to pr
ovide Q.O.S. levels that only mainframes can provide today. And if you
didn't hea
r me mention so-called 'clustering' solutions currently available from
unnamed ve
ndors,
it's because they can't actually deliver these things -- not true Q.O.S.
That's
my opinion, anyway. Using a cluster to hide the fact that the underlying
systems
crash regularly is an extremely dangerous way to manage a computing
environment.
I'm going to cheat a bit and also give you my #2 feature-wish: I want
native fil
esystem replication. I don't care a whit about common server-based disk
store: yo
u don't get reliability or scaleability that way. I want to see distributed
(repl
icated,
not partitioned) filesystems that are transactionally coherent, to go along
with
the process-migration of course :-)
3. Soft-Update seems to be one step further than Journaling, it is the
"modern"
way of doing journaling, and FreeBSD has that feature. However, do you have
plans
to add to the FreeBSD fs some of the features found on XFS or JFS systems?
Matt Dillon: I would not characterize soft-updates as being a step-ahead of
jour
naling. At least not meta-data journaling. It's just another way of doing
things.
Even though soft-updates can theoretically perform better then even
meta-data jo
urnalling
the plain fact of the matter is that linear disk bandwidth has at least 25x
the
throughput of a random seek/write. So journaling meta-data has a fairly
small per
formance impact if you can asynchronize everything *else*. Softupdates
works extr
emely well
for UFS but the softupdates concept can break down with other filesystems -
it c
ould very well be impossible to implement softupdates-like operation on a
filesys
tem which implements directories as BTree's or hashes, for example. On
the other
hand
softupdates can commit meta-data operations out of order while still
maintaining
filesystem integrity, and it can do it in an infinitely fine-grained
fashion whi
ch naturally leads to better parallelism. Journaled filesystems typically
can't d
o that. So
the usefullness of the theory depends heavily on what your goals are. For
genera
l purpose work both theories work equally well.
Most filesystem-specific 'super' features are highly specialized and not
actuall
y useful in the vast majority of system installations. XFS has data
zoning featur
es and (at least under IRIX) the ability to guarentee data stream latency
and ban
dwidth. I
can count the number of applications that actually need those features on
one ha
nd with a few fingers cut off. XFS's major advantage, as with all journaled
files
ystems, is instant crash recovery. All else being equal this is a journaled
files
ystem's
biggest advantage for general purpose computing but, even so, supplying the
prop
er options to newfs when creating a UFS filesystem can drop fsck times by
an orde
r of magnitude on large filesystems. People using UFS are not really at
that much
of a
disadvantage. You can't provide any sort of Q.O.S. if you depend on fast
crash r
ecovery to be fast. Q.O.S. means having redundant hardware at the very
least. I c
an't comment on JFS, I've never used it.
All of the BSD camps make stability priority #1 and performance priority
#2. Per
formance and fast crash recovery is completely irrelevant if the filesystem
corru
pts the data or causes a crash! This is especially true as HD capacities
increase
and
filesystems become larger. I have never quite understood why the Linux
community
gets so revved up by the huge number of filesystems they support. As if
the shee
r number combine together to provide a more effective system! You don't get
relia
bility,
performance, and long term stability by playing with filesystems, you get
it by
choosing or focusing on one or two filesystems that deliver those
characteristics
. Depending on filesystem-specific 'super' features makes code non-portable
and i
s not
usually a good idea.
In anycase, most BSD developers are happy with UFS. Oh, when I say UFS I
really
mean UFS+FFS or UFS+FFS+SOFTUPDATES. UFS is not the ancient creaking
beast that s
ome people have stereotyped it as. The basic theory and structure was sound
and i
s still
sound to this very day. Over the years we've fixed bugs (what few bugs we
find),
added capability support, better caching, reorganized the layout in a
backwards-
compatible fashion, re-introduced reblocking (basically on-the-fly
defragmentatio
n),
softupdates, snapshot support, etc etc etc.
4. After the open source bubble bursted recently, a lot of companies seized
supp
ort and stoped contributing code to both Linux or BSD. How has this
affected the
BSD development?
Matt Dillon: It creates a short term disruption for the people involved
in regar
ds to their ability to contribute but I do not believe company layoffs will
have
any effect on the open-source movement itself or on Linux and BSD
development in
the long
term. The biggest contributors to open-source are not staple employees of a
comp
any who are hired specifically to interact with the open-source community.
They a
re people who have a real interest and love of open-source who happen to be
worki
ng at a
company in a leverageable position.
While there have been BSD related layoffs, it's nothing that was unexpected
and
has had much less of an impact on us then I'm sure the huge number of
linux-centr
ic companies going bust has had on the Linux psyche. All I can say is: It
aint ou
r (the
open-source community) fault. Most of the linux centric companies were
leeching
off the linux name, and those that weren't didn't fail because they were
using Li
nux, they failed because they didn't have a business model with a chance in
hell
of (ever)
going profitable. Open-source operates behind the scenes far more then it
operat
es in the public eye, and it's hard to sell support to hackers who actually
have
*fun* trying to figure out a problem. In some respects Linux and the BSDs
are poo
r
commercialization candidates because they are *too* good... that they
simply do
not require the level of support that something like Windows-NT or Oracle
might r
equire in a back-office setting.
Open source has created far more disruption and change in commercial
interests t
hen the other way around. I think it has been for the better, though I'm
sure man
y commercial entities (such as MS) aren't too happy about being forced to
be more
honest
with their customers. (hmm... actually I think they still haven't learned,
and l
ook at the effect. MS has gotten its fingers burned so many times in
their dirty
war against open-source that even long-time commercial partners don't
believe wha
t they say
any more!).
5. How do you feel that Linux got most of the attention the last couple
of years
, and it was able to move a bit faster to the desktop arena? Is the Desktop
marke
t interest at all the FreeBSD people?
Matt Dillon: I find it to be an interesting exercise in social engineering,
econ
omics, and psychology. Oh, you want to know what I *really* think?
I think biggest winner here is open-source. A great deal of what people
label as
'Linux' isn't actually Linux. It's open-source that compiles just as
easily on F
reeBSD (*without* linux emulation) as it does on Linux. Take GNOME and
KDE for ex
ample. No
linux emulation necessary there! The areas where FreeBSD has problems are
almost
entirely relegated to commercial binary-only distributions. Now, that
said, Linu
x is certainly the largest driver of interest that leads to the development
of ma
ny of
these projects. I don't think we would have GNOME or KDE without Linux.
As a dri
ver of interest Linux has earned its place at the top of heap.
In regards to the desktop... well, I'm not sure exactly what you are
asking. Bot
h Linux and FreeBSD are in the same boat there... the only way to drive
desktop a
cceptance is to ship machines pre-installed with the OS (whatever OS) and
preconf
igured
with a desktop so when you turn the thing on, you are ready to rock. The
only wa
y to do that is for the PC vendors to pre-install Linux (or FreeBSD, or
whatever)
.
Other then that common issue, there really is no difference between FreeBSD
and
Linux in regards to the desktop. Oh, we could integrate the sound a
little better
and it would be nice to get a native OpenGL implementation working, but
everythi
ng else is
already there, because both platforms are running the same GUI software.
6. Please explain to us what SMPng (next-generation symmetric
multi-processing)
and KSE (kernel scheduler entities) are, which are features to be found
on the BS
D-5-Current.
Matt Dillon: SMPng is FreeBSD's fine-grained mutex, interrupt threading,
and Gia
nt-removal implementation. Potentially kernel pre-emption is also part of
the equ
ation but the jury is still out on that. The purpose is to be able to
have severa
l mainline
processes and/or interrupts operating in kernel mode simultaniously. This
is the
primary scaleability issue in any SMP system. The work being done here
is roughl
y compareable to the SMP work being done in Linux. Linux is about a year
ahead of
us but
both Linux and the BSDs have a great deal of work to do to catch up with
Solaris
.
KSE is a totally new (but old idea) way of implementing userland threads.
The id
ea here is two fold: (1) to remove any requirement that userland code
understand
which system calls might block and which system calls might not block.
(2) to do
all
primary thread scheduling and switching in userland, where any given cpu
can swi
tch between threads with approximately the same overhead as a userland
subroutine
call.
With KSEs if a userland process makes a system call which blocks, the
kernel wil
l detach the kernel context (which is now blocked) and return directly to
the use
r mode scheduler using an 'upcall'. The userland scheduler can then
immediately s
witch to
another thread. Another system call will be given a new, fresh, KSE to play
with
. The blocked kernel context runs completely asynchronously from the
userland pro
cess until it finishes and can potentially run concurrently with other
detached K
SEs for
the same process. When a KSE completes the kernel notifies the userland
schedule
r allowing the userland scheduler to reschedule the 'blocked' thread
which is now
'returning' from the system call that originally blocked.
The essential difference between KSEs and both select/kqueue-based
threads and r
fork based threads is that with KSEs you get all the parallelism of the SMP
box a
nd all the power of a userland-only context switch between threads (read:
*very*
fast
switch times) without *any* of the kernel overhead. A program can literally
be r
unning thousands of threads with no significant kernel overhead. Only
blocked sys
tem calls eat kernel resources. In addition to this, we can manage kernel
resourc
es in the
face of thousands of threads by limiting the 'pool' of KSEs we assign to
any giv
en process or user or whatever. So if 500 of those 1000 threads block in
a syscal
l we just get a little less cpu-efficient and don't blow out kernel memory.
Currently FreeBSD can use both select/kqueue and rfork (linux-style)
threading.
KSEs bring us to the next level.
7. From the technical point of view, how would you rate the Linux 2.4
kernel com
pared to BSD's?
Matt Dillon: I don't know enough about recent linux kernels to be able to
rate t
hem, nor would it be P.C. I do follow the VM work being done in Linux and
in part
icular Rik van Riel's work. I think Linux is going through a somewhat
painful tra
nsition as
it moves away from a Wild-West/Darwinist development methodology into
something
a bit more thoughtful. I will admit to wanting to take a clue-bat to some
of the
people arguing against Rik's VM work who simply do not understand the
difference
between
optimizing a few nanoseconds out of a routine that is rarely called
verses spend
ing a few extra cpu cycles to choose the best pages to recycle in order
to avoid
disk I/O that would cost tens of millions of cpu cycles later on. It is
an attitu
de I had
when I was maybe 16 years old... that every clock cycle matters no matter
how it
s spent. Bull!
8. How is the "relationship" between the FreeBSD programmers and the
OpenBSD/Net
BSD ones? Do you share code, opinions, chatting regularly? Or all these BSD
proje
cts are completely independant to each other?
Matt Dillon: The BSD groups are like high school social circles. No,
really! Tha
t's the best analogy I can think of! Many developers focus on just their
little c
lique but a good chunk run in multiple circles. There are developers that
maintai
n the same
driver code across several BSD distributions. There are developers who
focus the
ir work in one BSD distribution but have ties to developers in others. If
the wor
k is interesting enough, such as the 'dirpref' work, developers that
focus on cod
ing in
other BSD distributions will pick up the patch set and bring it in. That is
how
FreeBSD got the dirpref code. Kirk imported it from OpenBSD into
FreeBSD-current
and I MFC'd it to -stable after it had been proven out in -current.
In many respects this development methodology gives us the best of both
worlds.
Developers are free to focus on the distribution they are most familar with
and i
f the work is interesting enough it gets several eyes from the other
distribution
s who not
only port the code in, but also review it. Testing can wind up occuring
in all t
he distributions simultaniously and with something like 'dirpref', if
someone fin
ds a bug it will almost certainly wind up being fixed in the other
distributions
within a
few days. Security bugs are independantly verified but often the fix is
common t
o all the BSDs and no duplicate work need occur. There is constant
borrowing goin
g on between the BSDs and even between BSD and Linux, especially in regards
to dr
iver code.
9. What is your opinion on .NET and do you think that it may be possible
that .N
ET change the OS "map" as we know it?
Matt Dillon: I believe .NET is Vapor. It's a marketing term dreamed up by
Micros
oft that will magically morph into whatever Microsoft eventually winds up
deliver
ing. MS announces grandiose ideas with cute catch phrases all the time, and
as wi
th any
good vapor there is always some basis in truth (if only a little pinprick).
The
reality is a little different though... remember, these are the people that
hyped
windows-ME up the wazoo and all we got out of it was a
speech-synthesized window
s
installation wizard! These are the people that called NT the unix-killer
and tol
d people it was as reliable as UNIX. .NOT is probably a more descriptive
term for
.NET. My guess is that it will turn into Microsoft-proprietary
rent-a-service gl
ue, and
that it will introduce an order of magnitude more security issues then IIS.
10. Some say that FreeBSD has the best VM ever, whem compared to any
other Opera
ting System. Do you think that there is still space for improvement and are
there
still features to be added?
Matt Dillon: I think we made great progress stabilizing the VM system and
workin
g out performance issues related to machine scaling in the -4.x series of
FreeBSD
releases. The machines have proven to be great workhorses in a wide
range of app
lications
and are able to provide the long term stability and performance required by
its
users. Generally speaking, the technology behind the VM system is quite
sound and
does not need much more in the way of improvement. Obviously in -5.x we
will be
multi-threading pieces of it for SMPng, but the core algorithms appear to
extend
cleanly to MP and 64 bit platforms and we do not expect to have to make
any fund
amental changes. There is always room for improvement, of course! While
we are li
kely to
stand pat with the VM core in early 5.x releases, there is a great deal
of work
planned to improve the I/O and buffer cache subsystems a little later on.
My pers
onal goal is to eventually remove the buffer cache entirely or at least
morph it
into
nothing more complex than an I/O staging subsystem.
Itojun, NetBSD Core Team
1. How is the "relationship" between the NetBSD programmers and the
OpenBSD/Free
BSD ones? Do you share code, opinions, chatting regularly? Or all these BSD
proje
cts are completely independant to each other?
Itojun: Yes we do chat with each other and share code/opinions. Some of the
deve
lopers do have commit access (can modify source code tree) for multiple
BSDs.
2. Do you incorporate code to NetBSD from OpenBSD or FreeBSD when important
chan
ges are made to these OSes?
Itojun: Yes, but depending on the characteristics of the changes. If it
is a one
-line change for security issue, we'd integrate them right away. If it is a
big f
eature addition, we review them carefully and sometimes do integrate the
changes,
sometime
do not (we get similar changes from others, we implement it ourselves, or
integr
ate it with lot of improvements).
3. What goodies the next version of NetBSD is scheduled to bring us?
Itojun: SMP (for multiple platforms!) and fine-grained thread support are
the bi
ggest targets we are attacking. More platforms support, of course.
4. NetBSD's goal is to port the OS to as many platforms as it can. Which
platfor
ms are still needed NetBSD to be ported and it is a priority to do so?
Itojun: Sony PlayStation2 (port exists, needs integration).
Theo de Raadt, OpenBSD Founder
1. How is the "relationship" between the OpenBSD programmers and the
FreeBSD/Net
BSD ones? Do you share code, opinions, chatting regularly? Or all these BSD
proje
cts are completely independant to each other?
Theo de Raadt: There are no formal relationships of any kind. That said,
since i
t is a free world, there are numerous developers who do talk to their
counterpart
s in the other group. Even when that does not happen, public mailing
lists and th
e mainstay
product of our projects -- source code -- is completely visible. What
more could
one want?
2. Do you incorporate code to OpenBSD from NetBSD or FreeBSD when important
chan
ges are made to these OSes?
Theo de Raadt: Sure, why wouldn't we?
3. What goodies the next version of OpenBSD is scheduled to going us?
Theo de Raadt: First off, I should reiterate what I have been saying for
5 years
: OpenBSD development is not revolutionary, but evolutionary. That means
that bet
ween one release and another, not a lot of big things happen, but instead
we shou
ld view it
as a series of about 10,000 - 20,000 small changes. Over a series of
OpenBSD rel
eases, this amounts to a very big deal. Any release from 2 years back feels
very
different from the current codebase we have, but actually labelling the big
chang
es between
two consecutive releases is very difficult. Thousands of these changes
are bug f
ixes, minor conformance improvements... things which I would argue matter
MUCH MO
RE than "new features".
That said, this next release has one big thing that people are waiting to
try ou
t: We have written a whole new packet filter / nat engine, and fully
integrated i
t into the system. People who are used to ipf will find that pf is much
like ipf,
but has
some improvements which we have always wanted to make (and which the old
ipf lic
ense had blocked us from doing).
The alpha port has been significantly improved to support many of the
higher end
models (kind of funny considering the entire platform is now end of
lifed...), a
nd we will be releasing our first ultrasparc beta.
Other than that and the thousands of little fixes and improvements
everywhere, a
nd probably a bunch of other things I have already forgotten,
4. OpenBSD's goal is to bring ultimate security to a server. By patching
the hol
es and only accepting proved software do you think that it keeps your
development
moving slow from implementing something new to the OS level and
releasing it pre
tty fast?
Theo de Raadt: No, I think it does not affect or release schedule or
development
process.
--
※ 来源:·BBS 水木清华站 smth.org·[FROM: 210.74.136.33]
--
※ 来源:·洪水天堂BBS站 bbs.floodheaven.org·[FROM: 地狱灌水使者jjksam]
※ 来源:·荔园晨风BBS站 bbs.szu.edu.cn·[FROM: 203.93.19.1]
[回到开始]
[上一篇][下一篇]
荔园在线首页 友情链接：深圳大学 深大招生 荔园晨风BBS S-Term软件 网络书店